Actually, rather than the Elasticsearch patch, I'd like you to try a couple patches to the search indexing code. Turns out it was more inefficient than I had thought. Alternatively, I could send you a build. Just need to know the Python version and whether you used easy_install or pip to install Review Board (or some other method).
Christian -- Christian Hammond President/CEO of Beanbag <https://www.beanbaginc.com/> Makers of Review Board <https://www.reviewboard.org/> On Mon, May 1, 2017 at 1:28 PM, Christian Hammond <[email protected]> wrote: > Sure thing. I realized though that out of the box, this won't work without > a patch to Review Board (the proper Elasticsearch support has only been > thoroughly tested in 3.0). How comfortable do you feel applying patches? > > Christian > > > -- > Christian Hammond > President/CEO of Beanbag <https://www.beanbaginc.com/> > Makers of Review Board <https://www.reviewboard.org/> > > On Mon, May 1, 2017 at 1:17 PM, Daniel Kuecker <[email protected]> wrote: > >> Thanks for the quick reply Christian! >> >> I would like the info on configuring Elasticsearch if you don't mind. >> >> --DK >> >> On Monday, May 1, 2017 at 3:12:06 PM UTC-5, Christian Hammond wrote: >>> >>> Hi Daniel, >>> >>> I believe this happens because unlike review requests, we have no way of >>> determining with the search indexing stuff we use if a user's information >>> is "new" to the index. Basically, we have no "last updated" for the users. >>> >>> I do see an area in the search index for users that isn't as optimal, so >>> we can probably shave down the number of queries a fair amount, which will >>> help. >>> >>> I assume you're using the default search backend of Whoosh? If so, at >>> your scale, it might be time to install Elasticsearch. We can give you some >>> info on how to configure that in settings_local.py (the upcoming Review >>> Board 3.0 makes this very easy -- just select it as your search backend and >>> fill in fields). Indexing with Elasticsearch is more efficient for large >>> amounts of data. >>> >>> Christian >>> >>> -- >>> Christian Hammond >>> President/CEO of Beanbag <https://www.beanbaginc.com/> >>> Makers of Review Board <https://www.reviewboard.org/> >>> >>> On Mon, May 1, 2017 at 12:59 PM, Daniel Kuecker <[email protected]> >>> wrote: >>> >>>> I have a fresh 2.5.9 install with a db taken from a backup of another >>>> 2.5.9 install (basically a backup of my production instance). >>>> >>>> On the production instance, I am experiencing 100% cpu usage every time >>>> the update_index cron job runs causing my users to complain about >>>> responsiveness of the site. >>>> >>>> On my production system, the cron job shows it is indexing all the >>>> records in the user table while only the new records in the review requests >>>> table. >>>> >>>> For debugging, I restored the production db backup to my dev machine >>>> and set up a 2.5.9 instance pointing to it. When update_index runs on the >>>> dev box (given the "-v 3" arg), I get the following warning: >>>> >>>>> >>>>> WARNING:py.warnings:/usr/lib/python2.7/site-packages/haystack/indexes.py:167: >>>>> UserWarning: No updated date field found for 'User' - not restricting by >>>>> age. >>>>> warnings.warn(update_field_msg) >>>>> Indexing 1971 users >>>>> indexed 1 - 1000 of 1971 (by 16819). >>>>> indexed 1001 - 1971 of 1971 (by 16819). >>>> >>>> >>>> It then proceeds to index the entire user table and only the changed >>>> review requests just as my production instance does. >>>> >>>> I have run the rebuild_index command on both systems. >>>> >>>> Am I missing something? I have the cron job set to run ever 1/2 hour >>>> and am about to have to turn search indexing off to ensure the site is >>>> responsive. >>>> >>>> --DK >>>> >>>> >>>> >>>> -- >>>> Supercharge your Review Board with Power Pack: >>>> https://www.reviewboard.org/powerpack/ >>>> Want us to host Review Board for you? Check out RBCommons: >>>> https://rbcommons.com/ >>>> Happy user? Let us know! https://www.reviewboard.org/users/ >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "reviewboard" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> Supercharge your Review Board with Power Pack: >> https://www.reviewboard.org/powerpack/ >> Want us to host Review Board for you? Check out RBCommons: >> https://rbcommons.com/ >> Happy user? Let us know! https://www.reviewboard.org/users/ >> --- >> You received this message because you are subscribed to the Google Groups >> "reviewboard" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/ Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/ Happy user? Let us know! https://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups "reviewboard" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
