I’m still out of ideas for the moment. But as for real-time indices, this is a good place to start: https://freelancing-gods.com/thinking-sphinx/real_time.html <https://freelancing-gods.com/thinking-sphinx/real_time.html>
— Pat > On 12 Mar 2018, at 2:56 am, mrmanishs <[email protected] > <mailto:[email protected]>> wrote: > > Thanks Pat for the follow-up. > > Yes, was happening on each and every request. As I stated, it only happened > through Passenger also, not through console. > > I think I'll switch to real-time indices, if you think of anything else, let > me know. It's got me stumped. > > On Sunday, March 11, 2018 at 12:12:56 AM UTC-5, Pat Allan wrote: > I know it’s hard to test now that you’ve disabled deltas, but I wondering… > did it happen on every single request? > > It’s just that, previously - as already noted - failed indexing didn’t raise > exceptions. If two requests came at the same time that both made alterations > to the same model, one of the delta processing attempts would work, the other > wouldn’t - but the end result is still both alterations would be in Sphinx > (the successful processing captured both changes). > > Now, in the same scenario, one would work, and the other would fail and raise > an exception. The data would still be persisted in Sphinx (and given deltas > are invoked in after_commit, in the database as well), but the 500 error > resulting from the failure is a terrible user experience. > > If this is the situation, then the best solution is to either shift delta > processing into a background worker (via ts-sidekiq-delta, ts-delayed-delta, > or ts-resque-delta), or to use real-time indices. > > Of course, if this isn’t the case - if a single request raises the exception, > and there’s no other concurrent requests altering the same model, then the > cause will be something else (and I’m really not sure what it could be!). > Perhaps real-time indices are worth investigating either way? They’ve been > around in TS for a good while now and I use them for any of my new projects > that need Sphinx, as do significantly sized Flying Sphinx customers. > >> On 11 Mar 2018, at 2:39 pm, mrmanishs <[email protected] <>> wrote: >> >> Thanks for getting back Pat. Yes deltas were working great on the old server. >> >> The permission is 644 and owner is root. >> >> On Saturday, March 10, 2018 at 7:45:48 PM UTC-5, Pat Allan wrote: >> So, one thing that’s probably changed in that upgrade is that the newer >> versions of Riddle are now a little fussier about running indexing commands, >> and raise exceptions depending on the status code of the `indexer` command. >> So, it could be possible this issue was always happening but the errors >> weren’t apparent. I feel that’s unlikely though - you’d probably have >> noticed if delta results weren’t appearing? >> >> Also worth considering is the underlying error: >> >>> FATAL: failed to lock >>> /sites/cap/classifieds.acceleweb.com/db/sphinx/production/address_delta.spl: >>> <http://classifieds.acceleweb.com/db/sphinx/production/address_delta.spl:> >>> Resource temporarily unavailable, will not index >> >> I presume the path there is correct? What are the permissions on that file? >> Is the owner root? >> >> — >> Pat >> >>> On 10 Mar 2018, at 1:12 am, mrmanishs <[email protected] <>> wrote: >>> >>> Thanks. I actually migrated to a new server and installed the latest >>> version of Sphinx/TS. >>> >>> New server: Sphinx 3.0.2, TS 3.4.2 on Rails 5.1.5 >>> Old server: Sphinx 2.2.6, TS 3.2.0 on Rails 4.1.6 >>> >>> Could I have missed something in the upgrade that these symptoms may show? >>> >>> I have disabled deltas for now to get it going, but it's not ideal. >>> >>> >>> On Friday, March 9, 2018 at 8:58:02 AM UTC-5, Pat Allan wrote: >>> Hmm. Have you tried running a rebuild? And has this problem started only >>> recently? >>> >>> As a stop-gap solution, you could disable deltas and run the `ts:index` >>> task more regularly, but granted, that’s not ideal. >>> >>>> On 9 Mar 2018, at 2:36 pm, mrmanishs <man...@ <>acceleweb.com >>>> <http://acceleweb.com/>> wrote: >>>> >>>> >>>> I have nginx in front of passenger, so all the passenger log entries go >>>> into the nginx one. The only item in nginx is the url calls come back with >>>> a 500. >>>> >>>> Thanks for the note. >>>> >>>> Any other ideas? Also, up for doing some real-time debugging if it would >>>> help. I am not really sure what's going on and it's impacting >>>> functionality on site :( >>>> >>>> >>>> >>>> On Thursday, March 8, 2018 at 5:14:47 PM UTC-5, Pat Allan wrote: >>>> Hmm. Is there anything related to this issue in your passenger logs? >>>> And/or if you’re using nginx/apache/etc in front of that, their logs? >>>> >>>> Unrelated, but: indexed_models is no longer needed/used by TS v3+. >>>> >>>> — >>>> Pat >>>> >>>>> On 9 Mar 2018, at 1:04 am, mrmanishs <[email protected] <>> wrote: >>>>> >>>>> Thanks Pat. >>>>> >>>>> - searchd and nginx are both being run as root. Everything's root >>>>> actually... >>>>> - I have searchd in /usr/local/bin and confirmed the production bin_path >>>>> is /usr/local/bin . I have this in my thinking_sphinx.yml file for prod: >>>>> >>>>> production: >>>>> mysql41: 9401 >>>>> configuration_file: "<path_to_config>" >>>>> log: "<path_to_log>" >>>>> query_log: "<path_to_query_log>" >>>>> pid_file: "<path_to_pid_file>" >>>>> min_infix_len: 3 >>>>> bin_path: '/usr/local/bin' >>>>> indexed_models: >>>>> - Address >>>>> quiet_deltas: true >>>>> >>>>> Any other thoughts? >>>>> >>>>> On Wednesday, March 7, 2018 at 11:45:43 PM UTC-5, Pat Allan wrote: >>>>> Hi Manish, >>>>> >>>>> I’m at a conference today, so please excuse the brevity, but it sounds >>>>> like this might be a permissions issue. See this note in the >>>>> documentation: >>>>> https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas >>>>> <https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas> >>>>> (Both the deltas section and possibly the following section for passenger) >>>>> >>>>> Certainly, real-time indices shouldn’t hit these issues, but you should >>>>> certainly be able to get your current setup working. >>>>> >>>>> Hope this helps! >>>>> >>>>> — >>>>> Pat >>>>> >>>>>> On 8 Mar 2018, at 2:58 pm, mrmanishs <[email protected] <>> wrote: >>>>>> >>>>>> After further investigation, it looks like an issue with delta indexing. >>>>>> Is there a fix? Should real-time indices be used instead, is that a >>>>>> better approach? >>>>>> >>>>>> On Wednesday, March 7, 2018 at 4:39:16 PM UTC-5, mrmanishs wrote: >>>>>> Whenever I try to do anything in my app through passenger, I get an >>>>>> error similar to this: >>>>>> >>>>>> F, [2018-03-07T21:35:13.949318 #30762] FATAL -- : >>>>>> [d6484d23-620d-4f33-8f17-13c12b7dab53] Riddle::CommandFailedError >>>>>> (Sphinx command failed to execute): >>>>>> F, [2018-03-07T21:35:13.949426 #30762] FATAL -- : >>>>>> [d6484d23-620d-4f33-8f17-13c12b7dab53] >>>>>> F, [2018-03-07T21:35:13.949536 #30762] FATAL -- : >>>>>> [d6484d23-620d-4f33-8f17-13c12b7dab53] >>>>>> app/controllers/listing_controller.rb: >>>>>> >>>>>> >>>>>> That's all I get... no other information. >>>>>> >>>>>> The same instructions work in console and there are no issues. >>>>>> Everything else on the rails app works fine. >>>>>> >>>>>> Any idea? What else can I do to get more information? >>>>>> >>>>>> Manish >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Thinking Sphinx" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to thinking-sphi...@ <>googlegroups. >>>>>> <http://googlegroups.com/>com <http://googlegroups.com/>. >>>>>> To post to this group, send email to thinkin...@ <>googlegroups.com >>>>>> <http://googlegroups.com/>. >>>>>> Visit this group at https://groups.google.com/group/thinking-sphinx >>>>>> <https://groups.google.com/group/thinking-sphinx>. >>>>>> For more options, visit https://groups.google.com/d/optout >>>>>> <https://groups.google.com/d/optout>. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "Thinking Sphinx" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to [email protected] <>. >>>>> To post to this group, send email to [email protected] <>. >>>>> Visit this group at https://groups.google.com/group/thinking-sphinx >>>>> <https://groups.google.com/group/thinking-sphinx>. >>>>> For more options, visit https://groups.google.com/d/optout >>>>> <https://groups.google.com/d/optout>. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Thinking Sphinx" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected] <>. >>>> To post to this group, send email to [email protected] <>. >>>> Visit this group at https://groups.google.com/group/thinking-sphinx >>>> <https://groups.google.com/group/thinking-sphinx>. >>>> For more options, visit https://groups.google.com/d/optout >>>> <https://groups.google.com/d/optout>. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Thinking Sphinx" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] <>. >>> To post to this group, send email to [email protected] <>. >>> Visit this group at https://groups.google.com/group/thinking-sphinx >>> <https://groups.google.com/group/thinking-sphinx>. >>> For more options, visit https://groups.google.com/d/optout >>> <https://groups.google.com/d/optout>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Thinking Sphinx" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <>. >> To post to this group, send email to [email protected] <>. >> Visit this group at https://groups.google.com/group/thinking-sphinx >> <https://groups.google.com/group/thinking-sphinx>. >> For more options, visit https://groups.google.com/d/optout >> <https://groups.google.com/d/optout>. > > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at https://groups.google.com/group/thinking-sphinx > <https://groups.google.com/group/thinking-sphinx>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/thinking-sphinx. For more options, visit https://groups.google.com/d/optout.
signature.asc
Description: Message signed with OpenPGP
