Re: [ts] Customising SQL Query

[Pat Allan]

Hi Simone,

I'm sorry for the slow response.

It sounds like you may have already found the `:sql` option for search calls to 
pass through a limited set of SQL-based options as part of populating the 
search results:
https://freelancing-gods.com/thinking-sphinx/v5/searching.html#advanced

Granted, that's quite old, and the syntax was inspired by ActiveRecord's pre v3 
approach.  But there isn't anything else of this manner in Thinking Sphinx.

I realise it may not be ideal, but I would recommend instead querying Sphinx 
for the instance ids (using Article.search_for_ids instead of Article.search), 
and then - as/when needed - perform the SQL query using those ids. This avoids 
the initial SQL query to populate the search results as ActiveRecord objects. 
Would this be feasible in your situation?

Kind regards,

--
Pat
On 7 Nov 2022 at 9:51 PM +1100, Simone Miller , wrote:
> Hi Pat
>
> Hope you're keeping well.
>
> We're using a thinking sphinx search and we'd like to chain additional sql 
> generating active record queries on to the search. Ideally, we wouldn't want 
> to run the query or load the models immediately, so that we can maintain a 
> consistent code flow to instances where a search is not applied.
>
> We've seen we can define sphinx scopes and provide options within the search 
> itself - e.g. sql, select, with etc. Would there be a way to do this from the 
> outside - e.g. Article.search('pancakes').any_further_queries (that weren't 
> sphinx scopes)?
>
> Many thanks
> Simone
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/d2a693ca-6169-444c-a7bf-5ca23c7f590cn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/f2d4ed46-4914-416c-84e4-34a9f940023a%40Spark.

Re: [ts] (encoded) ? character in the search string produces no results

[Pat Allan]

Thanks Walter - a delayed response, but very happy to see you’ve got the 
solution all figured out without my input  and glad to know TS is still 
largely working well for you! :)

> On 2 Aug 2022, at 10:23 pm, Walter Lee Davis  wrote:
> 
> Thanks again, Pat, for making such a wonderful tool for all of us to enjoy as 
> if it was a toaster we bought at Target. I needed to upgrade Manticore on the 
> server. Once I did, and stumbled through enough of the post upgrade process 
> necessary to get the Rake task to work, I was able to rebuild my index and 
> start the server and all's well with the world. 
> 
> For anyone who comes along after me, the key to getting the server to work 
> was to turn it off in systemd (apt had put it back on), delete the old config 
> folder in /etc/manticoresearch (I just nuked that entire folder), and then 
> use the Rake task to rebuild the indices. Everything Just Worked™.
> 
> Walter
> 
>> On Aug 2, 2022, at 4:34 PM, Walter Lee Davis  wrote:
>> 
>> Oh, fascinating. I am using manticore locally, but sphinxsearch on the 
>> server. That might explain why this works perfectly on my Mac...
>> 
>> Walter
>> 
>>> On Aug 2, 2022, at 4:08 PM, Walter Lee Davis  wrote:
>>> 
>>> Following up: the search results controller is as follows:
>>> 
>>> ```
>>> class ResultsController < ApplicationController
>>> def index
>>>  @results = []
>>>  if params[:q].present?
>>>@results = 
>>> ThinkingSphinx.search(ThinkingSphinx::Query.escape(params[:q].to_s), 
>>> :excerpts => {
>>>  # :limit=> 1000,
>>>  :around => 40,
>>>  :force_all_words => true,
>>>  :chunk_separator => '  '
>>>  }, page: params[:page])
>>>@results.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
>>>  end
>>> end
>>> end
>>> 
>>> ```
>>> 
>>> This is virtually unchanged for years.
>>> 
>>> Walter
>>> 
 On Aug 2, 2022, at 4:00 PM, Walter Lee Davis  wrote:
 
 I am using ThinkingSphinx latest, with Manticore fairly recent, and an 
 upgraded Rails 7 application. If the details matter a lot, I can dig in 
 and find them. But I have discovered an odd edge case. When the user 
 enters a search string that ends with a ? (which gets properly encoded in 
 the form submission as %3F), the search returns no results. When that 
 character is stripped off, I get lots of results.
 
 There are no errors in Rails logging related to this, both return a 200. 
 The no results shows my regular "no results" page, and there is no further 
 detail from the search engine (although I do not know off-hand where to 
 tail its logs).
 
 Please let me know if there is anything I can do of a debugging nature to 
 make this easier to diagnose. This site has been running for a very long 
 time, and only recently upgraded to the latest version of TS along with 
 the Rails 7 upgrade. No further changes were made to the site beyond the 
 modernization of the configuration etc. as outlined in the Wiki.
 
 Thanks in advance,
 
 Walter
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 "Thinking Sphinx" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to thinking-sphinx+unsubscr...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/thinking-sphinx/73E33C92-C658-46E2-826F-1EEE831617E7%40wdstudio.com.
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Thinking Sphinx" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to thinking-sphinx+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/thinking-sphinx/D988AA4A-EE80-4998-B908-325B0C3D945C%40wdstudio.com.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphinx+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/thinking-sphinx/5F9CCD8A-2EAC-49C2-85A7-D894590E18A3%40wdstudio.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/5182FB49-106F-43C9-B6C7-564F5B72E9A4%40wdstudio.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: [ts] Unexpected results using without condition

[Pat Allan]

Hi Mike,

I’m a little baffled by these results too. Does it work properly if you send 
through just the single integer value (rather than array):

  MyModel.search(without: {my_integer: 3})

(Of course even if this works, if you’re wanting to pass through arrays of 
values then it’s not so helpful)

Cheers,

— 
Pat

> On 23 May 2022, at 7:17 pm, mikej  wrote:
> 
> I’m having some unexpected results:
> 
> results = MyModel.search(nil,without: {my_integer: [3]})
> results.map(&: my_integer)
> [1, 1, 1, 2, 1, 3, 1, 1, 1, 2, 1, 3, 1, 1, 1, 2, 1, 3, 1, 1]
> 
> I was expecting:
> [1, 1, 1, 2, 1, 1, 1, 1, 2, 1, 1, 1, 1, 2, 1, 1, 1]
> 
> ThinkingSphinx::Index.define :my_model, with: :active_record, delta: true do
>   …..
>   has my_integer
> end
> 
> Any help greatly appreciated,
> 
> Thanks in advance,
> 
> Mike
> 
> 
> Tried with
> Sphinx 2.2.11
> thinking-sphinx-4.2.0
> 
> And
> 
> Sphinx 2.3.3
> thinking-sphinx-5.4.0
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/fe62a51b-05fd-4447-bb06-d16c9f74fd5cn%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/1BDD8817-7E38-4568-A531-77C3821D9890%40freelancing-gods.com.

Re: [ts] Reusing Thinking Sphinx SQL connection to utilize CALL KEYWORDS

[Pat Allan]

That’s a good question! Something like the following should do the trick:

ThinkingSphinx::Connection.take do |connection|
  connection.execute “CALL KEYWORDS(‘word*’, ‘model_core’)"
end

Do let me know if there’s further questions.

Cheers,

— 
Pat

> On 7 Dec 2021, at 4:37 am, 'jason...@me.com' via Thinking Sphinx 
>  wrote:
> 
> I'm trying to add autocomplete to a keyword search field. I'm using Manticore 
> 4.0.2, and wish to use the CALL KEYWORDS syntax to get normalized keywords 
> from the index, which is built with min_prefix_len=2 and index_exact_words=1.
> 
> How can I use Thinking Sphinx's current mysql connection to fetch this result 
> set?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/add483ea-4431-4111-8a17-e8c1f7a8ca88n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/61AF7BC6-1724-41DC-9D07-7E1079929BDB%40freelancing-gods.com.

Re: [ts] Indexing tag names

[Pat Allan]

If you’ve got tag names and their corresponding ids, I think it’d be better 
(and more accurate) to query Sphinx by the ids:

  # in the index:
  has tag_ids

  # when searching, maybe something like:
  tag = Tag.find_by(name: params[:tag_name])
  Document.search params[:query], :with => {:tag_ids => tag.id}

It doesn’t answer the question why octothorps aren’t being indexed/searched 
correctly, but this should mean better search results generally.

Cheers,

— 
Pat

> On 24 Feb 2021, at 1:48 am, Walter Lee Davis  wrote:
> 
> 
> 
>> On Feb 23, 2021, at 12:02 AM, Pat Allan > <mailto:p...@freelancing-gods.com>> wrote:
>> 
>> Having the setting in the default block should be fine - you should be able 
>> to see the charset_table setting in the generated Sphinx configuration files.
>> 
>> Also: I generally recommend just using ts:rebuild, as that handles both 
>> real-time indices and SQL-backed indices (i.e. it’s running the same things 
>> as ts:rt:rebuild) - if you’re finding ts:rebuild is not working well for 
>> you, I’m keen to hear why!
> 
> While I was fighting with this, and fiddling with the configuration to use 
> has instead of indexes, I got myself into a state where ts:rebuild would blow 
> up with a SQL error (I think it was a Sphinx SQL error) and ts:rt:rebuild 
> would work fine. But with the current configuration that I shared with you, 
> both work.
> 
>> 
>> All that said, doesn’t sound like you’re doing anything wrong. I wonder if 
>> html_strip is somehow filtering out the octothorps? Though I’m pretty sure 
>> it’s looking just for HTML tags… still, may be worth turning that off to 
>> double-check.
>> 
>> And I’ve just run some quick tests locally - without the custom 
>> charset_table value, I find the string “#test” is found by Sphinx when 
>> searching by “#test” or “test” (because # is ignored, given it’s not an 
>> indexable character - so the two searches are actually identical). Adding in 
>> the charset_table setting, rebuilding - searching for #test returns a 
>> result, but test doesn’t (as that now doesn’t exist as a standalone word in 
>> what’s indexed).
>> 
>> I doubt it matters, but: which version of Sphinx are you using?
> 
> Sphinx 2.2.11-id64-release (95ae9a6), TS 5.0.0.
> 
> It's definitely odd. I'm not sure if re-indexing is picking up the tag names 
> when it runs en masse, and it seems to be something with GutenTag. If I find 
> a document in console, the object that I get back has tag_names set to nil, 
> but if I then call tag_names on that object, I get back the array of strings 
> I am expecting. It's just the value that I see inside the <> brackets 
> initially when to_s is called on the found object by irb, so I don't know if 
> that's significant at all, or is getting in the way of Sphinx extracting the 
> values. Again, when I test in console by calling my tags_for_indexing method 
> on a found object, I get back the expected string value.
> 
> I've told the client that she may need to get rid of her beloved hashtags in 
> the tagging interface, or use Gutentag in place of Sphinx to get "everything 
> tagged with this tag". I'm not convinced that's a bad idea, either.
> 
> Walter
> 
>> 
>> — 
>> Pat
>> 
>>> On 23 Feb 2021, at 3:10 pm, Walter Lee Davis  wrote:
>>> 
>>> Thanks for the speedy reply. I tried adding the charset table as 
>>> recommended, but I am not seeing any difference in my search results. I did 
>>> differ from the directions slightly, in that I put the character set in the 
>>> default block at the top of my Yaml file, since it's then included in all 
>>> of the environments. I figured that should work, but in case it doesn't can 
>>> you explain why?
>>> 
>>> default: 
>>> morphology: stem_en
>>> html_strip: true
>>> batch_size: 300
>>> charset_table: "0..9, A..Z->a..z, _, a..z, U+410..U+42F->U+430..U+44F, 
>>> U+430..U+44F, U+23"
>>> 
>>> development:
>>> <<: *default
>>> 
>>> test:
>>> <<: *default
>>> 
>>> production:
>>> <<: *default
>>> 
>>> staging:
>>> <<: *default
>>> mysql41: 9320
>>> 
>>> 
>>> I forced a full rebuild/reindex with rake ts:rt:rebuild. When that didn't 
>>> seem to change things, I also ran rake ts:rebuild. My understanding is that 
>>> the first of these should be done when you use the Real Time index. If I'm 
>>> mistaken, please let me know.
>>> 
>>> 

Re: [ts] Indexing tag names

[Pat Allan]

Having the setting in the default block should be fine - you should be able to 
see the charset_table setting in the generated Sphinx configuration files.

Also: I generally recommend just using ts:rebuild, as that handles both 
real-time indices and SQL-backed indices (i.e. it’s running the same things as 
ts:rt:rebuild) - if you’re finding ts:rebuild is not working well for you, I’m 
keen to hear why!

All that said, doesn’t sound like you’re doing anything wrong. I wonder if 
html_strip is somehow filtering out the octothorps? Though I’m pretty sure it’s 
looking just for HTML tags… still, may be worth turning that off to 
double-check.

And I’ve just run some quick tests locally - without the custom charset_table 
value, I find the string “#test” is found by Sphinx when searching by “#test” 
or “test” (because # is ignored, given it’s not an indexable character - so the 
two searches are actually identical). Adding in the charset_table setting, 
rebuilding - searching for #test returns a result, but test doesn’t (as that 
now doesn’t exist as a standalone word in what’s indexed).

I doubt it matters, but: which version of Sphinx are you using?

— 
Pat

> On 23 Feb 2021, at 3:10 pm, Walter Lee Davis  wrote:
> 
> Thanks for the speedy reply. I tried adding the charset table as recommended, 
> but I am not seeing any difference in my search results. I did differ from 
> the directions slightly, in that I put the character set in the default block 
> at the top of my Yaml file, since it's then included in all of the 
> environments. I figured that should work, but in case it doesn't can you 
> explain why?
> 
> default: 
>  morphology: stem_en
>  html_strip: true
>  batch_size: 300
>  charset_table: "0..9, A..Z->a..z, _, a..z, U+410..U+42F->U+430..U+44F, 
> U+430..U+44F, U+23"
> 
> development:
>  <<: *default
> 
> test:
>  <<: *default
> 
> production:
>  <<: *default
> 
> staging:
>  <<: *default
>  mysql41: 9320
> 
> 
> I forced a full rebuild/reindex with rake ts:rt:rebuild. When that didn't 
> seem to change things, I also ran rake ts:rebuild. My understanding is that 
> the first of these should be done when you use the Real Time index. If I'm 
> mistaken, please let me know.
> 
> Thanks again!
> 
> Walter
> 
>> On Feb 22, 2021, at 10:51 PM, Pat Allan  wrote:
>> 
>> Hi Walter,
>> 
>> I’m pretty sure Sphinx doesn’t index punctuation by default. If you want 
>> octothorps included, you’ll need to define a custom charset_table value (per 
>> environment in `config/thinking_sphinx.yml`) which includes that character. 
>> The Sphinx docs outline the default, so best to take that and then add in 
>> the octothorp (U+23).
>> http://sphinxsearch.com/docs/current.html#conf-charset-table
>> https://freelancing-gods.com/thinking-sphinx/v5/advanced_config.html#character-sets-and-tables
>> 
>> Keep in mind that this will impact all uses of that character in all fields 
>> - there’s no way to have it apply to just some fields (or, in this case, 
>> words that only start with that character).
>> 
>> Once you’ve added this configuration, a full rebuild will be required.
>> 
>> Cheers,
>> 
>> — 
>> Pat
>> 
>>> On 23 Feb 2021, at 2:41 pm, Walter Lee Davis  wrote:
>>> 
>>> I'm using GutenTag to apply tags to individual pages in a CMS. The Document 
>>> model uses TS5 with Real-Time Indexing. I've set up my index thusly:
>>> 
>>> # in the model
>>> def tags_for_indexing
>>>   tag_names.join ' '
>>> end
>>> 
>>> # in the index
>>> ThinkingSphinx::Index.define :document, :with => :real_time do
>>> scope { Document.where(id: Document.publicly.map{ |d| 
>>> [d.id].concat(d.descendants.published.map(&:id)) }.flatten) }
>>> 
>>> indexes title
>>> indexes teaser
>>> indexes body_html
>>> indexes author_display
>>> indexes tags_for_indexing
>>> 
>>> has created_at, type: :timestamp
>>> has updated_at, type: :timestamp
>>> end
>>> 
>>> I've tested the method, and confirm that it outputs a space-delimited 
>>> string of words for the tags.
>>> 
>>> I run rake ts:rt:rebuild and everything seems to go fine. But trying to 
>>> search on some of these tag names is not returning the results I am 
>>> imagining. The client has insisted on making some of these tags start with 
>>> an octothorp, because she is writing about "hashtags" on Twitter. Most tags 
>>> do not have punctuation in them. I am able to find other terms, even very 
&

Re: [ts] Indexing tag names

[Pat Allan]

Hi Walter,

I’m pretty sure Sphinx doesn’t index punctuation by default. If you want 
octothorps included, you’ll need to define a custom charset_table value (per 
environment in `config/thinking_sphinx.yml`) which includes that character. The 
Sphinx docs outline the default, so best to take that and then add in the 
octothorp (U+23).
http://sphinxsearch.com/docs/current.html#conf-charset-table 

https://freelancing-gods.com/thinking-sphinx/v5/advanced_config.html#character-sets-and-tables
 


Keep in mind that this will impact all uses of that character in all fields - 
there’s no way to have it apply to just some fields (or, in this case, words 
that only start with that character).

Once you’ve added this configuration, a full rebuild will be required.

Cheers,

— 
Pat

> On 23 Feb 2021, at 2:41 pm, Walter Lee Davis  wrote:
> 
> I'm using GutenTag to apply tags to individual pages in a CMS. The Document 
> model uses TS5 with Real-Time Indexing. I've set up my index thusly:
> 
> # in the model
>  def tags_for_indexing
>tag_names.join ' '
>  end
> 
> # in the index
> ThinkingSphinx::Index.define :document, :with => :real_time do
>  scope { Document.where(id: Document.publicly.map{ |d| 
> [d.id].concat(d.descendants.published.map(&:id)) }.flatten) }
> 
>  indexes title
>  indexes teaser
>  indexes body_html
>  indexes author_display
>  indexes tags_for_indexing
> 
>  has created_at, type: :timestamp
>  has updated_at, type: :timestamp
> end
> 
> I've tested the method, and confirm that it outputs a space-delimited string 
> of words for the tags.
> 
> I run rake ts:rt:rebuild and everything seems to go fine. But trying to 
> search on some of these tag names is not returning the results I am 
> imagining. The client has insisted on making some of these tags start with an 
> octothorp, because she is writing about "hashtags" on Twitter. Most tags do 
> not have punctuation in them. I am able to find other terms, even very 
> obscure ones, when I don't use punctuation in the tag names. 
> 
> Does this sound like something that I can fix, or should I advise the client 
> to lay off the octothorps?
> 
> Walter
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/EA71574B-9EBF-484E-A5FA-BF7CD53A10BC%40wdstudio.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/05B716CE-D5C7-40F6-BDE3-EC2859738632%40freelancing-gods.com.

Re: [ts] Realistic limit to # of indexed records?

[Pat Allan]

Hi Jeremy,

It’s great to hear that TS and Sphinx are still working well for you :)

In terms of expanding its scale in a reliable sense, I’ve a couple of thoughts:

You may want to consider real-time indices instead of SQL-backed indices. This 
removes the need for deltas and merging, and thus for full reindex calls. That 
said, this only works if all updates/inserts are done in ways that invoke 
ActiveRecord callbacks, as that’s how the real-time updates happen.
Sharding your larger models may also be an option? Especially if there’s clear 
boundaries between what’s being updated - the more static records can be in 
certain shards that don’t require the full reindex, whereas the more frequent 
changes are kept to other indices that get reprocessed daily. This would keep 
the reprocessing time down.

These two approaches could be used together, too - sharded real-time indices.

As to whether there’s an upper limit of records - not that I’m aware of, but 
I’m not the best person to ask. It may be worth asking on Sphinx’s own forum, 
and/or through the Manticore team’s channels as well (given they’re a fork of 
Sphinx that seems to be getting far more frequent updates - and it works as a 
drop-in replacement for Sphinx, so Thinking Sphinx doesn’t complain at all).

Hope this helps!

— 
Pat

> On 6 Feb 2021, at 7:28 am, 'jer...@shopittome.com' via Thinking Sphinx 
>  wrote:
> 
> We've been using Sphinx and ThinkingSphinx and Shphinx since 2008 and it's 
> always been amazing.
> 
> We generally have ~1m records indexed across a variety of fields and 
> attributes using it to filter users selections. It's super fast and works 
> great.
> 
> We also are using it in a larger scale and the records are now around 10m and 
> growing. Indexing itself is time consuming on this one. We're also using a 
> delta index and ts:merge but do need to re-index for updated caches once each 
> day.
> 
> Do we see an upper end to the number of records we can index?
> 
> Thanks! Jeremy
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/40b6f71c-bfd4-4788-a8c7-9a9d35ad7952n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/CD9B9D3E-3AF6-4330-8960-9546D6E4FED0%40freelancing-gods.com.

Re: [ts] Manticore? Should I jump ship from Sphinx?

[Pat Allan]

Hi Walter,

>From Thinking Sphinx’s perspective externally and internally, they are the 
>same. The binaries have the same names, which is because Manticore is a fork 
>of Sphinx. Manticore has been under active development for some time, so 
>there’s a bunch of new features there, but Thinking Sphinx doesn’t take 
>advantage of those, so there’s not much in the way of new benefits.

But the advantages for Manticore I see are as follows:

It works with MySQL v8
It’s under active development
It remains open source

Sphinx v3.1.1 does work with MySQL v8, but I’ve found that this release of 
Sphinx doesn’t like empty indices, and so it’s not viable for anyone using 
SQL-backed delta indices with Thinking Sphinx. Sphinx is currently not 
open-sourced - which I suspect was prompted by the appearance of Manticore, 
because it had been open before that - but it also doesn’t seem to be under 
active development. 3.1.1 was released 18 months ago, and there’s been nothing 
new since.

That’s not to say Sphinx no longer works - I’ve found v2.2.11 continues to work 
well, but given it must be compiled against what is an old version of MySQL 
(5.7 or older), that makes it a far less compelling option moving forward.

So it’s perhaps worth giving Manticore a shot, at least from the perspective of 
future-proofing your app. You’re not going to need to change anything in your 
use of Thinking Sphinx, though you will need to re-index from scratch - it’s 
recommended that you delete all Sphinx index and binlog files. There’s no new 
features available to Thinking Sphinx users though - I’m not against adding new 
things in, mind you, but there’s been no requests thus far.

Cheers,

— 
Pat

> On 21 Apr 2020, at 9:28 am, Walter Lee Davis  wrote:
> 
> I have a very old application, started in Rails 3, upgraded to 4, and now 
> being rebuilt in 6. I have read some of the docs for Manticore, and I think 
> heard it mentioned in this list not too long ago, too. Given that I have 
> never really exhausted what Sphinx can do, I've never considered changing for 
> change's sake. Can anyone give me the pro's and con's of the two? Should I 
> invest time in learning a new engine? Is there that much different to absorb? 
> I suspect that TS renders a lot of these differences quite moot, since it 
> straddles over the two and gives me a higher-level interface to configure and 
> consume either engine.
> 
> Thoughts?
> 
> Thanks in advance,
> 
> Walter
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/8651C796-908E-4FB4-ABDD-5C5F0BAAF3E5%40wdstudio.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/6CEC3882-E0B8-4E6C-8027-9E0BCE7FB3CB%40freelancing-gods.com.

Re: [ts] New error on an old app

[Pat Allan]

Hi Walter

I’m guessing you’ve upgraded to Thinking Sphinx 4.x at some point recently?

Due to Sphinx’s support for string-typed attributes (which have been around 
several years, but are relatively new compared to other types), Thinking Sphinx 
no longer automatically typecasts filters via the :with option from strings to 
integers (as they actually need to be strings) - and while this became optional 
in v3.4.0, it was enforced in v4.0.0 
.

All you’ll need to do in this specific case is ensure params[:set] is cast to 
an integer:

  :with => {:title_set => params[:set].to_i}

But of course, this will also apply to anywhere else you were passing string 
values (usually provided by the params hash) to any attribute filters (:with, 
:without, :with_all).

Cheers,

— 
Pat

> On 18 Apr 2020, at 6:36 am, Walter Lee Davis  wrote:
> 
> More detail about this. Here's the index:
> 
> ThinkingSphinx::Index.define :title, :with => :active_record do
>  set_property :group_concat_max_len => 10.megabytes
> 
>  indexes :title, :sortable => true
>  indexes teaser
>  indexes content.plain, :as => :plain_text
>  indexes author_name, :sortable => true
>  has roles(:person_id), :as => :people_ids 
>  has :id, :as => :title_id
>  has author_id, created_at, updated_at
>  has set, :as => :title_set
>  where sanitize_sql(["publish", true])
> end
> 
> The set column is an integer. 
> 
> The controller interprets this request thusly:
> 
>  def set
>@results = Title.search 
> "\"#{ThinkingSphinx::Query.escape(params[:q].to_s)}\"",
>:with => {:title_set => params[:set]},
>:page => params[:page],
>:star => true,
>:excerpts => {
>  :limit=> 1000,
>  :around => 40,
>  :force_all_words => true,
>  :chunk_separator => ''
>} rescue Kaminari::paginate_array []
>@results.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
>@hits = @results.total_entries rescue 0
>  end
> 
> Like I said, it's a very old app.
> 
> Walter
> 
>> On Apr 17, 2020, at 4:22 PM, Walter Lee Davis  wrote:
>> 
>> index title_core: unsupported filter type 'string' on int column - SELECT * 
>> FROM `title_core` WHERE MATCH('\"epicurus\"') AND `title_set` = '1851' AND 
>> `sphinx_deleted` = 0 LIMIT 0, 20; SHOW META):
>> 
>> This is coming right out of my logs today. It has not been a problem in the 
>> past.
>> 
>> Walter
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphinx+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/thinking-sphinx/33D96497-05FF-4816-9ADD-0A29CB80E0FE%40wdstudio.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/ED1246D3-EFEB-4DA3-A47E-1850C1D67B95%40wdstudio.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/C9DD7B96-7ADD-4DE0-AF7A-9EFE8E1ADDC2%40freelancing-gods.com.

Re: [ts] Successful MySQL2 installation for Sphinx.

[Pat Allan]

Great to hear, and thanks for sharing the solution! :)

— 
Pat

> On 3 Apr 2020, at 12:24 am, 'Justin Underwood' via Thinking Sphinx 
>  wrote:
> 
> Dear Pat Allan,
> 
> Thanks very much for your help, I think you are right. MySQL2 was affected by 
> an old MySQL installation.
> 
> For the record, this worked:
> 
> Removed all installations of MySQL and Sphinx
>   brew uninstall mysql, brew uninstall mysql@5.7
>   brew uninstall sphinx
> 
> Reinstalled MySQL@5.7 and Sphinx
>   brew install mysql@5.7
>   brew install sphinx
> 
> Ran 'gem install mysql2' from command line including the following '--with' 
> flags. I think the path to 'mysql_config' is what made this work. This seems 
> to correct the earlier error 
>   Cannot find mysql_config at /usr/local/bin/mysql_config
> when running gem install mysql2 without the flags.
> 
>   gem install mysql2 --
> --with-mysql-config=/usr/local/Cellar/mysql@5.7/5.7.29/bin/mysql_config 
> --with-cflags=\"-I/usr/local/opt/openssl/include\" 
> --with-ldflags=\"-L/usr/local/opt/openssl/lib\"
> 
> 
> Yours,
> Justin Underwood
> 
> 
> On Monday, March 30, 2020 at 12:30:43 PM UTC+1, Justin Underwood wrote:
> I am using macOS Mojave 10.14.6.
> 
> I am trying to configure Sphinx to use MySQL database for a Ruby on Rails 
> project.
> 
> I have been using Homebrew to install both Sphinx and MySQL. MySQL installs 
> correctly with Homebrew.
> 
> 
> I am experiencing errors running ./configure --with-mysql
> 
> I get the. error 
> 
> ERROR: cannot find MySQL include files.
> 
> Check that you do have MySQL include files installed.
> The package name is typically 'mysql-devel'.
> 
> If include files are installed on your system, but you are still getting
> this message, you should do one of the following:
> 
> 1) either specify includes location explicitly, using --with-mysql-includes;
> 2) or specify MySQL installation root location explicitly, using --with-mysql;
> 3) or make sure that the path to 'mysql_config' program is listed in
>your PATH environment variable.
> 
> To disable MySQL support, use --without-mysql option.
> 
> 
> How should the include files be linked to? Should these files be installed 
> manually. If so how.
> 
> Yours,
> 
> Justin Underwood
> 
> database creations.co.uk <http://creations.co.uk/>
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/02a30ee3-997b-4ab0-b734-302327b12be2%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/thinking-sphinx/02a30ee3-997b-4ab0-b734-302327b12be2%40googlegroups.com?utm_medium=email_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/0EF228D0-9476-414A-8CAE-6A71DDF958C5%40freelancing-gods.com.

Re: [ts] Cannot configure Sphinx to work with MySQL.

[Pat Allan]

Hi Justin,

I’m not quite sure I’m understanding the situation you’re dealing with here. 
Normally, If you’re installing Sphinx via Homebrew, you shouldn’t need to run 
“./configure …” - that’s only required as part of the process of compiling 
Sphinx manually.

But also: due to the most recent releases of Sphinx being closed source, 
they’re not available on Homebrew - but, none of the open-source releases 
(latest v2.2.11) support MySQL v8. So, it can take a bit of wrangling to get 
the appropriate versions all installed - but I believe Sphinx on Homebrew does 
have a dependency explicitly for MySQL 5.7, so that should work.

If you want to use a modern Sphinx release (v3.1.1), then you’ll need to 
download the binaries for that instead from the Sphinx website. However, I’ve 
found that it doesn’t allow empty indices (so, using it with SQL-backed indices 
and deltas isn’t an option) - I consider this a bug, but perhaps it’ll never be 
fixed. v3.1.1 was released 18 months ago, and there haven’t been any changes 
since.

A slightly different option is to use Manticore instead, which began as a fork 
of Sphinx but is under active development. Thinking Sphinx will work with 
Manticore (the related executables for both projects have the same names), so 
this may be a worthwhile alternative to explore.

The short version:

Want to use SQL-backed indices? Use Sphinx v2.2.11 and MySQL v5.7 (which is 
what Homebrew should install).
Want to use real-time indices? Sphinx v2.2.11 is fine, but you can also install 
the binaries for v3.1.1
Want to use a more active search project with Thinking Sphinx (with either 
SQL-backed or real-time indices)? Give Manticore a shot.

If any of these paths don’t work, do let me know.

Cheers,

— 
Pat

> On 30 Mar 2020, at 10:01 pm, 'Justin Underwood' via Thinking Sphinx 
>  wrote:
> 
> I am using macOS Mojave 10.14.6.
> 
> I am trying to configure Sphinx to use MySQL database for a Ruby on Rails 
> project.
> 
> I have been using Homebrew to install both Sphinx and MySQL. MySQL installs 
> correctly with Homebrew.
> 
> 
> I am experiencing errors running ./configure --with-mysql
> 
> I get the. error 
> 
> ERROR: cannot find MySQL include files.
> 
> Check that you do have MySQL include files installed.
> The package name is typically 'mysql-devel'.
> 
> If include files are installed on your system, but you are still getting
> this message, you should do one of the following:
> 
> 1) either specify includes location explicitly, using --with-mysql-includes;
> 2) or specify MySQL installation root location explicitly, using --with-mysql;
> 3) or make sure that the path to 'mysql_config' program is listed in
>your PATH environment variable.
> 
> To disable MySQL support, use --without-mysql option.
> 
> 
> How should the include files be linked to? Should these files be installed 
> manually. If so how.
> 
> Yours,
> 
> Justin Underwood
> 
> database creations.co.uk
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/e098a07e-dc4c-4efa-ae2b-9a295fb53100%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/22450D4D-AA62-4DBD-A845-A32FF7B6F4AF%40freelancing-gods.com.

Re: [ts] Matching words shorter than min_infix_length with stars: true

[Pat Allan]

I don’t have any suggestions for this I’m afraid. Sorry Roger!

Might be better to ask these questions on the Sphinx forum, or via 
StackOverflow or similar (in the latter case, you could tag it with Manticore 
as well, their team seems pretty on the ball about providing support - they’re 
a fork of Sphinx if you’re not familiar with it).

Cheers,

— 
Pat

> On 30 Sep 2019, at 8:20 pm, Roger Kind Kristiansen 
>  wrote:
> 
> Hi!
> 
> I'm using wildcard search and a min_infix_length: 2. This works great, but 
> not when I'm searching for an entry that contains a single-letter word, and I 
> search for that word. For example if the entry has a text of "a random name", 
> and I search for this exact text, the search string ends up being "'*a* 
> *random* *name*'", and the *a* ends up not matching anything.
> 
> Is there any simple way around this using thinking sphinx? Grateful for any 
> suggestions!
> 
> --
> 
> A couple of things I've considered:
> 
> * Manually set up the search string for each search, so it becomes "'a random 
> name | '*a* *random* *name*'". I'd rather avoid having to do this, if possible
> * Set index_exact_words=1. This seems to depend on 
> dictionaries/morphology/wordforms to work, and I'm not sure how I'd use that, 
> as the searches are in Norwegian
> 
> 
> Cheers,
> Roger
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/fbf92a3f-4fbe-44ff-82e6-147954f8%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/F310E18D-6C7C-4CE8-96F6-4ACCAB9CE6DE%40freelancing-gods.com.

Re: [ts] Using Multiple Languages in Single Database Field

[Pat Allan]

Hi Mike,

Sorry for not getting back to you on this promptly.

Here’s the code I’ve been testing with (within the TS test suite):

Article.create! :title => "ochrona środowiska"
index

search = Article.search("środowiska")
search.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane

expect(search.first.excerpts.title).
  to eq(%q{ochrona środowiska}

And, when I first ran it, it didn’t pass:

Failure/Error:
  expect(search.first.excerpts.title).
to eq(%q{ochrona środowiska})

  expected: "ochrona środowiska"
   got: "ochrona środowiska”

As you may notice, it’s the leading ś that doesn’t get matched correctly.

I tested this against Sphinx 2.2.11, 3.1.1, and Manticore 3.1.2, all failed. 
Then I did a bit of searching, and came across this post:
http://tswiackiewicz.github.io/inside-the-source-code/sphinxsearch/sphinxsearch-odmiana-wyrazen/
 


I can’t read Polish, but I took the suggested charset_table settings and added 
them to my Article index, and the test passes. So perhaps that’s worth adding 
to your app’s config? Either in config/thinking-sphinx.yml or on a per-index 
basis with set_property :charset_table => "..."

Mind you, I’m not across what each of those transformations cover - and you may 
want a more extensive set (as covered here: 
https://yob.id.au/2008/05/08/thinking-sphinx-and-unicode.html 
).

If this doesn’t help, though, do let me know!

— 
Pat

> On 16 Sep 2019, at 7:03 pm, mikej  wrote:
> 
> ŚRODOWISKA
> 
> Many thanks,
> 
> Mike
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/37d87243-2763-4f43-8c56-ff18ff38c5cc%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/0D5106A7-B943-497C-8426-F38024ADFF66%40freelancing-gods.com.

Re: [ts] Using Multiple Languages in Single Database Field

[Pat Allan]

Hi Mike,

Good to know the deltas issue is resolved :)

As for excerpts… can you give me an example of a word that’s not matching 
appropriately? I can then try reproducing it locally. It may end up being a bug 
in Sphinx itself, but I’m not yet sure.

Cheers,

— 
Pat

> On 10 Sep 2019, at 1:42 am, mikej  wrote:
> 
> Thanks Pat.
> 
> Yes UTF8 is good.  I only changed the collation to get the record saving at 
> all.
> 
> Right.  First problem solved, my muppetry.  The delta is working fine.  I was 
> saving a related record, so delta indexing was never going to be relevant.  
> This is fine.  Sorry.
> 
> I don't have any character/encoding related settings in 
> `config/thinking_sphinx.yml`.  I have upgraded to 2.2.11 but still have the 
> problem with the excerpts.  Not a disastrous situation, but would be good to 
> fix.
> 
> THANK YOU,
> 
> Mike
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/4bcefc1b-636c-40fb-be70-76dd5355b844%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/E00AF83B-EBCB-451E-9525-CD163730C649%40freelancing-gods.com.

Re: [ts] No success with wildcard search

[Pat Allan]

Hi Seb,

No need to apologise for the French! :)

You’ll likely need either min_infix_len or min_prefix_len set in 
config/thinking_sphinx.yml to ensure parts of words are indexed alongside full 
words:
https://freelancing-gods.com/thinking-sphinx/v4/advanced_config.html#infix-and-prefix-indexing
 


Walter’s suggestion about stemming is a good one, but I think stemming - for 
any language - is going to focus on a standard dictionary, rather than proper 
names? I’m not sure though, perhaps it takes a more general approach.
https://freelancing-gods.com/thinking-sphinx/v4/advanced_config.html#word-stemming--morphology
 


After changing any of these settings, you’ll need to rebuild your index to have 
the changes applied to Sphinx.

Give that a shot - in particular, the infix/prefix setting - and let us know if 
the problem persists!

Cheers,

— 
Pat

> On 17 Aug 2019, at 12:16 am, Walter Lee Davis  wrote:
> 
> I wonder if there’s some configuration needed to set the stemming language? 
> That’s just a guess. 
> 
> Walter
> 
> On Aug 16, 2019, at 8:42 AM, Seb A.  > wrote:
> 
>> Hi !
>> 
>> I'm stuck on a simple case and I can't really know why it doesn't work as 
>> expected.
>> 
>> I'm using a 5.2 rails app with TS 4.3.2 and Sphinx 3.1.
>> 
>> For my blog, here : https://milesandlove.com/photos 
>> , I'm indexing about 6000 images with only 
>> one field which is basically a concatenate of some fields :
>> 
>> ThinkingSphinx::Index.define :image, :with => :real_time do
>>   indexes searchable
>>   has rank, :type => :integer, :as => :rank_sort
>> end.
>> 
>> 
>> Got a searchable method in my image.rb model as follow :
>> 
>> def searchable
>> [title, tag_list.map(&:downcase).join(" "), of, country, region, 
>> city].flatten.join(" ")
>>   end
>> 
>> 
>> Thus , this searchable return this kind of content :
>> 
>> "Incroyables formations rocheuses paysages montagnes d'Allemagne Allemagne 
>> La Saxe Bastei" 
>> "Coucher de soleil, vallée de la mort manuelle deserts coucher de soleil du 
>> Chili Chili Nord du Chili Valle de la Luna" 
>> "Regard Granada Nicaragua manuelle du Nicaragua Nicaragua Granada et 
>> alentours Granada"
>> 
>> Now, my goal is to be able to search within this content with multiple 
>> words. 
>> 
>> If I try : "Seb Montagnes", it return results.
>> 
>> But if I try 'Seb Montagne", it return nothing, even with enable star set to 
>> true.
>> 
>> I can't understand how to make this work. Any idea? 
>> 
>> (as a note, sorry for the French words guys...)
>> 
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphinx+unsubscr...@googlegroups.com 
>> .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/thinking-sphinx/491ca9e5-c597-4836-9561-c93cd0cfffe1%40googlegroups.com
>>  
>> .
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/92C7F54A-8766-4F32-8D5E-5E1EDD6E7090%40wdstudio.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/thinking-sphinx/837F4E81-6394-4B51-9FF0-1AC086CBBA35%40freelancing-gods.com.

Re: [ts] Search results and matched term highlighting

[Pat Allan]

Hi Walter,

Sorry for the slow response… and to be honest, I don’t have a good answer for 
this behaviour. I’m really not sure what’s going on.

I did look over the available settings for excerpts:
http://sphinxsearch.com/docs/current.html#api-func-buildexcerpts 

… and anything that I feel would influence what you’re seeing (e.g. 
exact_phrase) defaults to what would be ideal in your site anyway.

I’m not sure if upgrading Sphinx would have any impact, but it may be 
worthwhile - at least to 2.2.11. That said, there’s nothing in the release 
notes for 2.2.10/11 that I can spot that suggests any change in behaviour.

If you really wanted to dig into it, I’d suggest building a test app that can 
reproduce the problem with a smaller dataset, and potentially share that here 
so I can have a look as well. Of course, it very much sounds like a Sphinx 
issue rather than anything to do with Thinking Sphinx, so whether I can 
actually fix things is not super likely.

Wish I could be more helpful!

— 
Pat

> On 9 Jul 2019, at 8:46 am, Walter Lee Davis  wrote:
> 
> I wonder if anyone knows how Sphinx goes about constructing the snippets that 
> are returned along with the matches to a search term. This page illustrates a 
> wild variety of examples of how one search term can be interpreted:
> 
> https://oll.libertyfund.org/search/results?q=power+corrupts
> 
> Note the first hit, from Alvis on Shakespeare. The exact phrase exists in the 
> third line of the snippet (on a desktop screen, YMMV). It is not highlighted. 
> In the third example, the result is from deep in the weeds of the footnotes, 
> and hits on the word power, and actually highlights it. The fifth hit gets 
> both power (twice) and corrupts, but misses the stem of corrupts in corrupt. 
> The second-to-the-last hit on that page, in Liberty, Order, and Justice, goes 
> on for several screens (208,135 words), with a single snippet that has grown 
> to encompass 725 individual keyword hits in one "paragraph" of source text.
> 
> I'm using Thinking Sphinx 3.1.2, and Sphinx is version 2.2.9
> 
> Here's the controller method that constructed this page:
> 
>@results = ThinkingSphinx.search 
> "\"#{ThinkingSphinx::Query.escape(params[:q].to_s)}\"",
>:page => params[:page],
>:star => true,
>:excerpts => {
>  :limit=> 1000,
>  :around => 40,
>  :force_all_words => true,
>  :chunk_separator => ''
>}.reject{ |r| r.class.to_s == 'NilClass' } rescue Kaminari::paginate_array 
> []
>@results.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
>@hits = @results.total_entries rescue 0
> 
> And these results are from mostly titles, but some pages. Here's the 
> definition for both:
> 
> # titles_index.rb
> ThinkingSphinx::Index.define :title, :with => :active_record do
>  set_property :group_concat_max_len => 10.megabytes
> 
>  indexes :title, :sortable => true
>  indexes teaser
>  indexes content.plain, :as => :plain_text
>  indexes author_name, :sortable => true
>  has roles(:person_id), :as => :people_ids 
>  has :id, :as => :title_id
>  has author_id, created_at, updated_at
>  has set, :as => :title_set
>  where sanitize_sql(["publish", true])
> end
> 
> #pages_index.rb
> ThinkingSphinx::Index.define :page, :with => :active_record do
> 
>  indexes :title, :sortable => true
>  indexes teaser
>  indexes body
>  has created_at, updated_at
> end
> 
> In the view, I'm using this tortured bit of ERB:
> 
><%= content_tag( :ol, 
> "#{result.excerpts.plain_contents}".gsub(/\s*<\/li>/,'').html_safe
>  ) if result.respond_to?(:plain_contents) %>
> 
> And there's no way to explain why some results are wrapped in the  class="match"> in the output from Sphinx, while others (nearby, in the same 
> set of results) are not.
> 
> Thanks in advance if anyone can enlighten me or point me toward documentation 
> of this feature. This is all very old code, maybe 6 or 8 years since I last 
> touched it. I've moved it to a newer server since I wrote all this, but 
> nothing much changed when I did that. My client would like to know, and I 
> don't have any good answers.
> 
> Walter
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To post to this group, send email to thinking-sphinx@googlegroups.com.
> Visit this group at https://groups.google.com/group/thinking-sphinx.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/thinking-sphinx/AAECECFD-619C-49AC-B4E7-63A6C87C2595%40wdstudio.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

Re: [ts] Re-using scopes from the indexed model

[Pat Allan]

Given I’m feeling ambivalent at best about such a change, I don’t think a PR is 
worth the time. Appreciate the discussion nonetheless!

> On 18 Mar 2019, at 1:40 am, Walter Lee Davis  wrote:
> 
> Not really. I guess it would just fall under POLS to allow it to work. The 
> documentation and method you have now is clear, and since there's no way I 
> can think of to allow for chained scopes with this new syntax, all it would 
> really be is a sugar for the lowest-hanging option. I could take a whack at a 
> PR for this if you think it makes sense, but I'm not entirely sold on its 
> necessity, either.
> 
> Walter
> 
>> On Mar 17, 2019, at 12:56 AM, Pat Allan  wrote:
>> 
>> Hi Walter,
>> 
>> This isn’t currently possible, though certainly it wouldn’t be hard to add 
>> that capability in, as you’ve noted. Right now I’m on the fence about 
>> whether I’d accept such a patch though - I like the idea of having one clear 
>> interface for such things (though I know I’ve not always followed that 
>> approach in the past!), and the only benefit I can think of at the moment is 
>> saving a few characters.
>> 
>> Are there other reasons around why this feature would be useful?
>> 
>> Cheers,
>> 
>> — 
>> Pat
>> 
>>> On 17 Mar 2019, at 6:34 am, Walter Lee Davis  wrote:
>>> 
>>> Would it be possible to allow the scope method in real time indexing to 
>>> accept the name of a scope on the indexed model? I'd like to be able to use
>>> 
>>> scope :published
>>> 
>>> rather than
>>> 
>>> scope { Speech.published }
>>> 
>>> I got an error indicating that scope expects a block (1 variable rather 
>>> than 0 passed) so it occurs to me that it may be possible to wrap the 
>>> current behavior in if block_given? and then drop back to 
>>> 
>>> the_inferred_model.call scope_argument
>>> 
>>> or [whatever actually works with models and scopes and isn't `send`]
>>> 
>>> Any thoughts about this?
>>> 
>>> Thanks,
>>> 
>>> Walter
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Thinking Sphinx" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to thinking-sphinx+unsubscr...@googlegroups.com.
>>> To post to this group, send email to thinking-sphinx@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/thinking-sphinx.
>>> For more options, visit https://groups.google.com/d/optout.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphinx+unsubscr...@googlegroups.com.
>> To post to this group, send email to thinking-sphinx@googlegroups.com.
>> Visit this group at https://groups.google.com/group/thinking-sphinx.
>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To post to this group, send email to thinking-sphinx@googlegroups.com.
> Visit this group at https://groups.google.com/group/thinking-sphinx.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Simplifying scope in real time index

[Pat Allan]

Hi Walter,

I can’t think of a better way to do this off the top of my head - I’m presuming 
there’s no special scope provided by Ancestry that allows access to all records 
(as I understand it, that’s what you’re after?) that can be used by 
find_in_batches?

— 
Pat

> On 17 Mar 2019, at 6:30 am, Walter Lee Davis  wrote:
> 
> I've got a combination of TS (latest) and Ancestry in play, and I'm just now 
> trying to move from SQL indices to real_time. I was able to make a tortured 
> bit of Ruby to collect all the IDs and then return them in a single 
> association that TS could find_in_batches over, but I am wondering if there 
> is a way to do this that I will not have to stare at so long in a few months, 
> when I've forgotten why it works:
> 
>  scope { Document.where(id: Document.publicly.map{ |d| 
> [d.id].concat(d.descendant_ids) }.flatten) }
> 
> publicly is a scope that gives me only the "roots" of the ancestry tree with 
> a combination of a particular set of slugs and the "published" flag, and then 
> I'm appending descendent_ids and making a new collection from the lot of them 
> with the outer "find". 
> 
> Any suggestions?
> 
> Thanks in advance,
> 
> Walter
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To post to this group, send email to thinking-sphinx@googlegroups.com.
> Visit this group at https://groups.google.com/group/thinking-sphinx.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Re-using scopes from the indexed model

[Pat Allan]

Hi Walter,

This isn’t currently possible, though certainly it wouldn’t be hard to add that 
capability in, as you’ve noted. Right now I’m on the fence about whether I’d 
accept such a patch though - I like the idea of having one clear interface for 
such things (though I know I’ve not always followed that approach in the 
past!), and the only benefit I can think of at the moment is saving a few 
characters.

Are there other reasons around why this feature would be useful?

Cheers,

— 
Pat

> On 17 Mar 2019, at 6:34 am, Walter Lee Davis  wrote:
> 
> Would it be possible to allow the scope method in real time indexing to 
> accept the name of a scope on the indexed model? I'd like to be able to use
> 
> scope :published
> 
> rather than
> 
> scope { Speech.published }
> 
> I got an error indicating that scope expects a block (1 variable rather than 
> 0 passed) so it occurs to me that it may be possible to wrap the current 
> behavior in if block_given? and then drop back to 
> 
> the_inferred_model.call scope_argument
> 
> or [whatever actually works with models and scopes and isn't `send`]
> 
> Any thoughts about this?
> 
> Thanks,
> 
> Walter
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To post to this group, send email to thinking-sphinx@googlegroups.com.
> Visit this group at https://groups.google.com/group/thinking-sphinx.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Has the Excerpt option changed?

[Pat Allan]

Hi Walter,

That warning definitely shouldn’t have been happening - I’d missed excerpts in 
the list of known options. I’ve just pushed a fix to the develop branch on 
GitHub, and will get a new release out in the next week or so.

Cheers,

— 
Pat

> On 1 Mar 2019, at 3:33 pm, Walter Lee Davis  wrote:
> 
> I am using TS again after many years, and so I have installed the latest 
> Sphinx 3 and the latest TS 4.1.0 with my Rails 5.1.6.1 project. I am getting 
> an odd response from Sphinx when I search:
> 
>  Sphinx  Unexpected search options: [:excerpts]
> 
> This is happening on Mac OS in development mode.
> 
> Here is my controller method:
> 
>@results = ThinkingSphinx.search(ThinkingSphinx::Query.escape(params[:q]), 
> :excerpts => {
>  :limit=> 1000,
>  :around => 40,
>  :force_all_words => true,
>  :chunk_separator => '  '
>})
>@results.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
> 
> And here is the index:
> 
> # frozen_string_literal: true
> 
> ThinkingSphinx::Index.define :document, :with => :active_record do
>  where sanitize_sql(["published", true])
> 
>  indexes title
>  indexes teaser
>  indexes body_html
>  indexes author
> 
>  has created_at, updated_at
> end
> 
> configuration:
> 
> indexer
> {
> }
> 
> searchd
> {
>  listen = 127.0.0.1:9306:mysql41
>  log = 
> /Volumes/eddy/Users/waltd/Documents/sandbox/lf-smith/log/development.searchd.log
>  query_log = 
> /Volumes/eddy/Users/waltd/Documents/sandbox/lf-smith/log/development.searchd.query.log
>  pid_file = 
> /Volumes/eddy/Users/waltd/Documents/sandbox/lf-smith/log/development.sphinx.pid
>  workers = threads
>  binlog_path = 
> /Volumes/eddy/Users/waltd/Documents/sandbox/lf-smith/tmp/binlog/development
> }
> 
> source document_core_0
> {
>  type = mysql
>  sql_host = localhost
>  sql_user = root
>  sql_pass = 
>  sql_db = lf-smith_development
>  sql_query_pre = SET TIME_ZONE = '+0:00'
>  sql_query_pre = SET NAMES utf8
>  sql_query = SELECT SQL_NO_CACHE `documents`.`id` * 4 + 0 AS `id`, 
> `documents`.`title` AS `title`, `documents`.`teaser` AS `teaser`, 
> `documents`.`body_html` AS `body_html`, `documents`.`author` AS `author`, 
> `documents`.`id` AS `sphinx_internal_id`, 'Document' AS 
> `sphinx_internal_class`, 0 AS `sphinx_deleted`, 
> UNIX_TIMESTAMP(`documents`.`created_at`) AS `created_at`, 
> UNIX_TIMESTAMP(`documents`.`updated_at`) AS `updated_at` FROM `documents` 
> WHERE (`documents`.`id` BETWEEN $start AND $end AND published) GROUP BY 
> `documents`.`id`, `documents`.`title`, `documents`.`teaser`, 
> `documents`.`body_html`, `documents`.`author`, `documents`.`id`, 
> `documents`.`created_at`, `documents`.`updated_at` ORDER BY NULL
>  sql_query_range = SELECT IFNULL(MIN(`documents`.`id`), 1), 
> IFNULL(MAX(`documents`.`id`), 1) FROM `documents`
>  sql_attr_uint = sphinx_deleted
>  sql_attr_bigint = sphinx_internal_id
>  sql_attr_timestamp = created_at
>  sql_attr_timestamp = updated_at
>  sql_attr_string = sphinx_internal_class
> }
> 
> index document_core
> {
>  type = plain
>  path = 
> /Volumes/eddy/Users/waltd/Documents/sandbox/lf-smith/db/sphinx/development/document_core
>  docinfo = extern
>  morphology = stem_en
>  html_strip = 1
>  source = document_core_0
> }
> 
> and core YAML file:
> 
> default: 
>  morphology: stem_en
>  html_strip: true
> 
> development:
>  <<: *default
> 
> test:
>  <<: *default
> 
> production:
>  <<: *default
> 
> 
> All of this is ripped straight from the documentation, as far as I can see. 
> Can you recommend any changes to make this work without that warning? Or do I 
> not need to worry about that? The search seems to work just fine.
> 
> Walter
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To post to this group, send email to thinking-sphinx@googlegroups.com.
> Visit this group at https://groups.google.com/group/thinking-sphinx.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Soft deleting users, default scopes

[Pat Allan]

Hi Simone,

Sorry for the delay in responding to this.

The default scope only has an impact when inserting/updating all records into 
Sphinx (via ts:index/ts:rebuild). If changes are made to a soft-deleted 
instance which fires the callback there, then the real-time processing will do 
its thing on just that record, and it’ll find itself back in Sphinx.

Also, as you’ve noted, ts:index does not delete existing records - it just 
updates existing ones and inserts new ones. A rebuild, on the other hand, 
clears out existing index data first.

If you’re not using a default scope, then you can set your real-time index’s 
scope to load only what you want:
https://freelancing-gods.com/thinking-sphinx/v4/real_time.html#eager-loading-associations
 


And to avoid callback behaviour in these soft-deleted instances, instead of 
using the default TS approach, you’ll want to have your own callback instead:
https://freelancing-gods.com/thinking-sphinx/v4/indexing.html#callbacks 


Hope all this helps! Do let me know if there are further questions.

Cheers,

— 
Pat

> On 6 Feb 2019, at 8:34 pm, Simone Miller  wrote:
> 
> Hi Pat
> 
> May I check in with you about an issue we're having with soft deleted users 
> being filtered out of searches? 
> 
> We're runnning thinking sphinx 4.0.0 with real time indices, soft deleting 
> users (without active record destroy callbacks) and have a default scope on 
> users { where(:deleted_at => nil) }
> We run 'flying-sphinx regenerate' each night - after which deleted users no 
> longer show up in searches. Are we right in thinking that having the default 
> scope means deleted users won't be added to the index?
> 
> We are going to remove the default scope! In the meantime, is there a way to 
> prevent thinking-sphinx from filtering out deleted users / unscoping our 
> default scope when the index is being generated? We're trying to avoid batch 
> updating deleted users to manually trigger real time callbacks.
> 
> We also noticed in development that running rake ts:index doesn't filter out 
> deleted users, but running rake ts:rebuild does. Could you explain why this 
> is?
> 
> Thank you
> Simone
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] ThinkingSphinx::SphinxError: Unknown MySQL error

[Pat Allan]

Hi Simone,

I’m pretty sure that the issue is an incompatibility between Sphinx 2.2.x 
releases (which is what Homebrew still uses) and MySQL 8 (which is also what 
Homebrew supplies). You’ll either need to downgrade MySQL to 5.7 (and then 
re-install Sphinx), or upgrade Sphinx to 3.1.1:
http://sphinxsearch.com/downloads/current/ 


In case anyone who comes across this post is interested, here’s the GitHub 
issue where the problem was identified (at least, from the perspective of 
Thinking Sphinx):
https://github.com/pat/thinking-sphinx/issues/1101 


TS v4 should work fine with Sphinx 3.1.1, though there may be some deprecation 
messages.

If this doesn’t fix things, do let me know!

Cheers,

— 
Pat

> On 4 Jan 2019, at 3:05 am, simhmil...@gmail.com wrote:
> 
> Hi Pat
> 
> Happy New Year. Am a little stuck with re-installing sphinx - if you had a 
> moment to help, that would be much appreciated.
> 
> We've been using thinking sphinx with rails. I've just re-installed mysql and 
> sphinx using brew, but am getting this error when seeding the database. We've 
> tried manually installing mysql and sphinx but not having much luck. Do you 
> have an idea of what it might be?
> 
> Thank you
> Simone
> 
> -- create_table("versions", {:force=>:cascade})
>-> 0.0127s
> -- add_foreign_key("activities", "snapshots")
>-> 0.0021s
> rake aborted!
> ThinkingSphinx::SphinxError: Unknown MySQL error
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/mysql2-0.5.2/lib/mysql2/client.rb:90:in
>  `connect'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/mysql2-0.5.2/lib/mysql2/client.rb:90:in
>  `initialize'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/connection/mri.rb:13:in
>  `new'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/connection/mri.rb:13:in
>  `client'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/connection/client.rb:28:in
>  `close!'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/connection.rb:28:in
>  `block in pool'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/innertube-1.1.0/lib/innertube.rb:88:in
>  `delete_element'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/innertube-1.1.0/lib/innertube.rb:140:in
>  `rescue in take'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/innertube-1.1.0/lib/innertube.rb:148:in
>  `take'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/connection.rb:36:in
>  `take'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/real_time/transcriber.rb:29:in
>  `block in copy'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in
>  `block in instrument'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in
>  `instrument'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in
>  `instrument'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/logger.rb:5:in
>  `log'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/real_time/transcriber.rb:28:in
>  `copy'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/real_time/callbacks/real_time_callbacks.rb:48:in
>  `block (2 levels) in persist_changes'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/real_time/callbacks/real_time_callbacks.rb:47:in
>  `each'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/real_time/callbacks/real_time_callbacks.rb:47:in
>  `block in persist_changes'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/real_time/callbacks/real_time_callbacks.rb:46:in
>  `each'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/real_time/callbacks/real_time_callbacks.rb:46:in
>  `persist_changes'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/thinking-sphinx-4.0.0/lib/thinking_sphinx/real_time/callbacks/real_time_callbacks.rb:13:in
>  `after_save'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:426:in
>  `block in make_lambda'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:236:in
>  `block in halting_and_conditional'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:517:in
>  `block in invoke_after'
> /Users/simone/.rvm/gems/ruby-2.3.1@hands/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:517:in
>  `each'
> 

Re: [ts] Specifying an index to update when running ts:index

[Pat Allan]

So happy to know you’re still finding the gem useful :)

— 
Pat

> On 20 Oct 2018, at 11:01 am, Jeremy Meigs  wrote:
> 
> OH my thanks Pat! I must be blind.
> 
> And yep... we're up to date with the latest. 
> 
> Thanks so much, this gem rocks and we've been using it for many years now.
> 
> Jeremy
> 
> 
> On Fri, Oct 19, 2018 at 4:56 PM Pat Allan  <mailto:p...@freelancing-gods.com>> wrote:
> Hi Jeremy,
> 
> This is possible using the INDEX_FILTER environment variable - as noted at 
> the very bottom of the indexing docs (yes, not super obvious I know!):
> https://freelancing-gods.com/thinking-sphinx/v4/indexing.html#processing 
> <https://freelancing-gods.com/thinking-sphinx/v4/indexing.html#processing>
> 
> However, that option is relatively recent - if you’re not using TS v3.4 or 
> newer, then a different approach is going to be required. Can you let us know 
> which version you are using?
> 
> Kind regards,
> 
> — 
> Pat
> 
>> On 20 Oct 2018, at 10:24 am, Jeremy Meigs > <mailto:jer...@shopittome.com>> wrote:
>> 
>> Anyone have an option for specifying an index so we don't rebuild them all. 
>> We have several indices and would like to specify which index to rebuild.
>> 
>> Thanks, Jeremy
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphinx+unsubscr...@googlegroups.com 
>> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
>> To post to this group, send email to thinking-sphinx@googlegroups.com 
>> <mailto:thinking-sphinx@googlegroups.com>.
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Specifying an index to update when running ts:index

[Pat Allan]

Hi Jeremy,

This is possible using the INDEX_FILTER environment variable - as noted at the 
very bottom of the indexing docs (yes, not super obvious I know!):
https://freelancing-gods.com/thinking-sphinx/v4/indexing.html#processing 


However, that option is relatively recent - if you’re not using TS v3.4 or 
newer, then a different approach is going to be required. Can you let us know 
which version you are using?

Kind regards,

— 
Pat

> On 20 Oct 2018, at 10:24 am, Jeremy Meigs  wrote:
> 
> Anyone have an option for specifying an index so we don't rebuild them all. 
> We have several indices and would like to specify which index to rebuild.
> 
> Thanks, Jeremy
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Sphinx on Elastic Beanstalk

[Pat Allan]

Hi Harjeet,

It looks like your installation of Sphinx has broken - it was apparently 
compiled with a version of the MySQL client libraries that is no longer 
present. Do you have any version of the MySQL client libraries installed? And 
how have you installed Sphinx?

Cheers

—
Pat

> On 27 Aug 2018, at 6:23 am, Harjeet Singh  <mailto:harje...@smartkidzclub.com>> wrote:
> 
> Hi Evan,
> 
> I am getting the same error and I searched all over for last 10 days and 
> simply cant get it to work. I am just using and EC2 instance with RDS.
> 
> I connect to RDS via EC2 security group on port 3306 to Index.
> 
> I was able to Index it once and the search is working with the index that was 
> created few weeks again.
> 
> When ever Is run indexer --rotate --all is get below error
> 
> I will really appreciate the help
> 
> best
> Harjeet
> 
> 
> 
> indexer: relocation error: indexer: symbol mysql_init, version 
> libmysqlclient_18 not defined in file libmysqlclient.so.18 with link time 
> reference
> 
> On Wednesday, March 28, 2018 at 2:33:02 PM UTC-4, Evan Shabsove wrote:
> Okay great!
> 
> I managed to get it all working! I can't believe it ahaha thanks for 
> everyone's help!
> 
> On Wednesday, March 28, 2018 at 11:46:34 AM UTC-4, Walter Davis wrote:
> Port and PID have literally nothing to do with one another. Sphinx uses port 
> 9312 by default, and you can configure it otherwise using the --port flag 
> when you start searchd.
> 
> Walter
> 
> > On Mar 28, 2018, at 11:36 AM, Evan Shabsove > 
> > wrote:
> >
> > So I have set up a separate server with my app's code running sphinx and 
> > have ran all the ts rake tasks.
> >
> > When configuring sphinx in my thinking_sphinx.yml file what address should 
> > I use? Currently I just have it pointing to the public IP of the server, 
> > but I would imagine I need to point it at the searchd client PID correct? 
> > Something like MY_IP:31096?
> >
> > Also what port should I open on my server?  Should it be port 31096?
> >
> > Thanks,
> > Evan
> >
> > On Wednesday, March 21, 2018 at 8:47:46 PM UTC-4, Pat Allan wrote:
> > No worries about asking the questions :)
> >
> > And yes, that thread contains the right advice - you want to deploy a copy 
> > of your app to the Sphinx server, and have the address setting configured 
> > with the appropriate IP/host. That server is where you’ll run all the TS 
> > rake tasks, and it’s where the Sphinx daemon is and the index files. Each 
> > of your app servers then make a request to this Sphinx server.
> >
> > If that’s not clear or if you have further questions, do ask :)
> >
> > —
> > Pat
> >
> >> On 22 Mar 2018, at 5:59 am, Evan Shabsove > wrote:
> >>
> >> I think you addressed the same problem in this thread as well 
> >> https://github.com/pat/thinking-sphinx/issues/1005 
> >> <https://github.com/pat/thinking-sphinx/issues/1005>
> >>
> >> Sorry if this comes up a lot, I bet you get these kind of questions all 
> >> the time.
> >>
> >> On Wednesday, March 21, 2018 at 2:46:18 PM UTC-4, Evan Shabsove wrote:
> >> Hey Pat, sorry I know this post was forever ago now but I was never quite 
> >> able to figure out a solution for this, currently we just turned searching 
> >> off on our app but that's far from desirable.
> >>
> >> I'm fairly new to the world of dev-ops so I don't think I was asking the 
> >> right question in my first message. Originally I was trying to install 
> >> Thinking Sphinx on every new instance launched by Elastic Beanstalk, but 
> >> this would be a poor way of doing it, correct? Because then every new 
> >> instance has different indexes?
> >>
> >> What I want to do now is set up a server that runs the Sphinx client, and 
> >> our app makes search requests to this server. This way searching is not 
> >> tied to the auto scaling environment.
> >>
> >> I guess my questions are is this possible? And if yes how can I tell my 
> >> rails app to point to this new server?
> >>
> >> Thanks for the help!
> >> Evan
> >>
> >> On Friday, January 5, 2018 at 10:07:02 AM UTC-5, Evan Shabsove wrote:
> >> Hmm okay, I'll try to get the client libraries installed, I'm a bit new to 
> >> yum as well so this process has been a bit of a challenge ahaha. I'll give 
> >> it a shot and let you know if that gets everything working, thanks!
> >>
> >> On Wednesday, January 3, 2018 at 9:27:30 PM UTC-5, 

Re: [ts] Surviving a server restart

[Pat Allan]

Hi Walter,

I’m not quite sure what the best approach is for your setup - getting things to 
run on boot on Linux machines is something I’m gradually getting the hang of 
(via scripts in /etc/init.d/), but perhaps your server is running Solaris? (I 
have a vague memory that is/was the platform of choice for Joyent) - so I’m 
less certain.

All that said: `ts:start` can be called whether Sphinx is running or not. If 
it’s already running, it won’t do anything (provided it can find the pid file 
reliably). So maybe a cron job that runs that is good enough?

Cheers,

—
Pat

> On 7 Aug 2018, at 9:56 am, Walter Lee Davis  wrote:
> 
> For the umpteenth time in the past 8 years this app has been in production, 
> the server restarted and TS failed to restart along with the rest of app. I 
> am using this very specific branch:
> 
>  remote: git://github.com/pat/thinking-sphinx.git
>  revision: de070904a8d1a5958be9d5d6ad162419cc443690
>  ref: de070904a8
>  branch: develop
>  specs:
>thinking-sphinx (3.1.2)
>  activerecord (>= 3.1.0)
>  builder (>= 2.1.2)
>  innertube (>= 1.0.2)
>  joiner (>= 0.2.0)
>  middleware (>= 0.1.0)
>  riddle (>= 1.5.11)
> 
> Which for historical reasons, I am loath to change. The server is a Joyent 
> SmartMachine, and I cannot change anything about it because the image is no 
> longer supported. I don't have a budget (although I am negotiating for one) 
> to really overhaul the hosting.
> 
> Can anyone suggest a way to (maybe with an initializer in the Rails app) get 
> the rake task ts:start to run when the app initializes, maybe checking first 
> to see if Sphinx is running? I have tried using a cron job to poke at it 
> periodically, but that seems to have stopped working too.
> 
> Thanks,
> 
> Walter
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To post to this group, send email to thinking-sphinx@googlegroups.com.
> Visit this group at https://groups.google.com/group/thinking-sphinx.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] Using hostname as address (docker)

[Pat Allan]

Hi Steve,

I’m not super familiar with Docker, so I’m not entirely sure what’s appropriate 
here, but a few thoughts that may help clarify the situation:

Are you trying to have Sphinx be available via a hostname, like your database? 
Or are you trying to have Sphinx talk to the database via its hostname (“db”)?

In the case of the former: Sphinx should accept hostnames as an alternative to 
IP addresses for the `address` setting. Are you finding that’s not the case?

In the case of the latter: Thinking Sphinx picks up the database settings from 
your ActiveRecord connection - thus, it should be using whatever’s in 
`config/database.yml`. Are you finding that’s not the case in the generated 
Sphinx configuration file?

In either case: you may want to ask in this GitHub issue, as others who are 
using Docker and Thinking Sphinx have been sharing their thoughts there:
https://github.com/pat/thinking-sphinx/issues/1010 


Also, from a general Sphinx/Thinking Sphinx configuration perspective: the 
mysql41 setting is what you should use for setting Sphinx’s port in TS v3+. The 
`port` setting is a legacy from TS v1/v2, and can make the Sphinx daemon 
confused - best to remove it.

Hope this helps! If you need further clarification, do ask :)

—
Pat

> On 1 Aug 2018, at 7:29 pm, Steve Redka  > wrote:
> 
> Hello
> 
> I'm trying to dockerize my application. It uses thinking sphinx and I have 
> trouble configuring it.
> 
> In particular, I can't configure it to have a reliable link to container with 
> database. This is my current config/thinking_sphinx.yml file:
>  development: 
>min_infix_len: 1
>charset_table: "0..9, english, U+0021..U+002F"
>port: 9306
>address: 172.17.0.1
># sql_host: db
># mysql41: 9306
>  production:
><<: *common
> 
> As you see, it uses direct ip to reference the container. It works on my 
> machine, but may break elsewhere, which ruins the whole reason to use docker. 
> So I want to reference container by its hostname, like it is already done in 
> database.yml:
> 
> common: 
>   adapter: mysql2
>   encoding: utf8
>   database: sociaball
>   username: <%= ENV['MYSQL_USER'] %>
>   password: <%= ENV['MYSQL_PASSWORD'] %>
>   host: db # Here
> 
> I tried using sql_host and host directives. Also, I tried putting 'db' into 
> clauses and using container name (sociaball_db_1).
> 
> So, is it possible to point to database server by hostname in 
> thinking_sphinx.yml?
> Is it even thinking sphinx issue?
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] ThinkingSphinx not connecting to remote Sphinx server

[Pat Allan]

Hi Cam,

Your current setup is fine (once you add a copy of the Rails app onto the 
database server) - especially if you’re using SQL-backed indices, because then 
Sphinx doesn’t have as much network overhead when it’s talking directly to the 
database.

If you’re using real-time indices, then it’s up to you. It might be more a 
matter of balancing the resources needed - I feel like the app server could be 
more demanding, so keeping things as you’ve got now is perhaps best? Certainly, 
I’d start with that and see how things go.

Cheers

— 
Pat

> On 18 Jun 2018, at 10:02 am, Cameron McKay  <mailto:cameron0...@gmail.com>> wrote:
> 
> Hey Pat,
> 
> Thanks for the quick reply!  At the moment my architecture consists of two 
> servers:  App Server (Rails app) and Database Server (Sphinx and MySQL).
> 
> Is this a best practice architecture to use with Sphinx?  Based on your 
> response, I am wondering if I should move Sphinx to the App Server or if I 
> should maintain a copy of the rails app on both servers.
> 
> Cheers, Cam.
> 
> On Sunday, 17 June 2018 21:57:44 UTC-5, Pat Allan wrote:
> Hi Cameron,
> 
> It looks like you’re running the rake task on the app server. The server with 
> Sphinx needs to have a copy of the Rails app, and *that* is where you run all 
> the Sphinx-related tasks (as there’s no way to start or index Sphinx 
> remotely).
> 
> Give that a shot, and let us know how you go!
> 
> Cheers,
> 
> — 
> Pat
> 
>> On 18 Jun 2018, at 9:51 am, Cameron McKay > wrote:
>> 
>> Hello,
>> 
>> I have Rails application running on the app server and I am connecting it to 
>> a remote server that contains MySQL and Sphinx using ThinkingSphinx.
>> 
>> I've updated my thinking_sphinx.yml file to point to the remote sphinx 
>> server.  The thinking_sphinx.yml configuration is shown in the image below:
>> 
>>  
>> <https://lh3.googleusercontent.com/-UPJexucayNM/Wycd2Wa_2OI/IvE/GpXQ2JiR-J0YDGFp0fvWiNzK-LaJZ59MwCLcBGAs/s1600/err2.JPG>
>> 
>> I'm running the command "bundle exec rake ts:start" and am getting the error 
>> shown in the image below.
>> 
>>  
>> <https://lh3.googleusercontent.com/-TxsKVON3pfY/WycdgFpdGHI/Iu8/pXxB5L5TBWsoJ2a3y6j_dGqvxE3rKzVUwCLcBGAs/s1600/err1.JPG>
>> 
>> I'm rather lost on this.  Any help would be greatly appreciated!
>> 
>> Cheers, Cam.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphi...@googlegroups.com <>.
>> To post to this group, send email to thinkin...@googlegroups.com <>.
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] ThinkingSphinx not connecting to remote Sphinx server

[Pat Allan]

Hi Cameron,

It looks like you’re running the rake task on the app server. The server with 
Sphinx needs to have a copy of the Rails app, and *that* is where you run all 
the Sphinx-related tasks (as there’s no way to start or index Sphinx remotely).

Give that a shot, and let us know how you go!

Cheers,

— 
Pat

> On 18 Jun 2018, at 9:51 am, Cameron McKay  > wrote:
> 
> Hello,
> 
> I have Rails application running on the app server and I am connecting it to 
> a remote server that contains MySQL and Sphinx using ThinkingSphinx.
> 
> I've updated my thinking_sphinx.yml file to point to the remote sphinx 
> server.  The thinking_sphinx.yml configuration is shown in the image below:
> 
>  
> 
> 
> I'm running the command "bundle exec rake ts:start" and am getting the error 
> shown in the image below.
> 
>  
> 
> 
> I'm rather lost on this.  Any help would be greatly appreciated!
> 
> Cheers, Cam.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] INDEX_FILTER env variable seems to have no effect?

[Pat Allan]

Hi Roger,

Sorry it’s taken me so long to reply, my inbox got a little out of control. And 
doubly so, because the answer, it turns out, was also my fault:

The documentation was wrong - you should include the _core or _delta suffix. 
I’ll fix the docs up - sorry for the confusion!

— 
Pat

> On 25 May 2018, at 10:46 pm, Roger Kind Kristiansen 
> mailto:roger.kristian...@gmail.com>> wrote:
> 
> When running "rake ts:index INDEX_FILTER=", thinking sphinx 
> goes on indexing all of them. I'm using the name of the index as defined in 
> :name parameter of the index definition, without _core or _delta, as 
> mentioned all the way at the bottom of this documentation. 
>  
> Just to make sure I've also tried defining the variable before running rake 
> ts:index.
> 
> Am I misunderstanding or messing something up here, or is there a bug? I'm on 
> v4.0.0.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

[ts] Newly Released: v4.0.0

[Pat Allan]

Hi all,

It’s been a long time between updates about released versions, but I’ve just 
published thinking-sphinx v4.0.0.

There’s not much in the way of breaking changes from the v3 releases (provided 
you’re up-to-date with v3.4.x, at least), and there’s some neat new features 
(merging support and UNIX sockets support), as well as allowing Flying Sphinx 
to use the standard ts-prefixed rake tasks (consistent behaviour across all 
platforms! Huzzah!)
… and if you are a Flying Sphinx customer, upgrading thinking-sphinx to v4.0.0 
and flying-sphinx to v2.0.0 is certainly recommended.

Slightly more detail can be found in a blog post:
https://freelancing-gods.com/2018/04/10/thinking-sphinx-v4.html 

And far more detail in the release notes:
https://github.com/pat/thinking-sphinx/releases/tag/v4.0.0 


Oh, and in case anyone here was curious - thinking-sphinx has been around for 
over 10 years!  Thank you so much to everyone here for using the gem, 
providing feedback, and contributing patches over the years :)

—
Pat

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] Sphinx on Elastic Beanstalk

[Pat Allan]

No worries about asking the questions :)

And yes, that thread contains the right advice - you want to deploy a copy of 
your app to the Sphinx server, and have the address setting configured with the 
appropriate IP/host. That server is where you’ll run all the TS rake tasks, and 
it’s where the Sphinx daemon is and the index files. Each of your app servers 
then make a request to this Sphinx server.

If that’s not clear or if you have further questions, do ask :)

—
Pat

> On 22 Mar 2018, at 5:59 am, Evan Shabsove <evan.shabs...@swtchev.com 
> <mailto:evan.shabs...@swtchev.com>> wrote:
> 
> I think you addressed the same problem in this thread as well 
> https://github.com/pat/thinking-sphinx/issues/1005 
> <https://github.com/pat/thinking-sphinx/issues/1005>
> 
> Sorry if this comes up a lot, I bet you get these kind of questions all the 
> time.
> 
> On Wednesday, March 21, 2018 at 2:46:18 PM UTC-4, Evan Shabsove wrote:
> Hey Pat, sorry I know this post was forever ago now but I was never quite 
> able to figure out a solution for this, currently we just turned searching 
> off on our app but that's far from desirable.
> 
> I'm fairly new to the world of dev-ops so I don't think I was asking the 
> right question in my first message. Originally I was trying to install 
> Thinking Sphinx on every new instance launched by Elastic Beanstalk, but this 
> would be a poor way of doing it, correct? Because then every new instance has 
> different indexes?
> 
> What I want to do now is set up a server that runs the Sphinx client, and our 
> app makes search requests to this server. This way searching is not tied to 
> the auto scaling environment.
> 
> I guess my questions are is this possible? And if yes how can I tell my rails 
> app to point to this new server?
> 
> Thanks for the help!
> Evan
> 
> On Friday, January 5, 2018 at 10:07:02 AM UTC-5, Evan Shabsove wrote:
> Hmm okay, I'll try to get the client libraries installed, I'm a bit new to 
> yum as well so this process has been a bit of a challenge ahaha. I'll give it 
> a shot and let you know if that gets everything working, thanks!
> 
> On Wednesday, January 3, 2018 at 9:27:30 PM UTC-5, Pat Allan wrote:
> Hi Evan,
> 
> I think you’re right about it being a dependency issue. Sphinx will require 
> mysql-related dependencies - you don’t need a MySQL server running, but you 
> will need client libraries installed. I’m not across yum, so I’m not sure 
> what’s ideal. Also, it’s worth noting that you may need to reinstall Sphinx 
> after the mysql dependencies, to ensure it picks up on them correctly.
> 
> Do let us know if you find a solution!
> 
> —
> Pat
> 
>> On 4 Jan 2018, at 8:29 am, Evan Shabsove <evan.s...@swtchev.com <>> wrote:
>> 
>> Hey everyone,
>> 
>> I've been trying to set up sphinx on an elastic beanstalk instance. I 
>> currently have searchd running, but when I try to run the command
>> 
>> RAILS_ENV=production bundle exec rake ts:index
>> 
>> I get an error of
>> 
>> using config file '/var/app/ondeck/config/production.sphinx.conf'...
>> WARNING: key 'enable_star' was permanently removed from Sphinx 
>> configuration. Refer to documentation for details.
>> WARNING: key 'enable_star' was permanently removed from Sphinx 
>> configuration. Refer to documentation for details.
>> indexing index 'custom_field_value_core'...
>> indexer: relocation error: indexer: symbol mysql_init, version 
>> libmysqlclient_18 not defined in file libmysqlclient.so.18 with link time 
>> reference
>> 
>> When I try to make a search query on the website I get an error of
>> 
>> "free":"no enabled local indexes to search
>> 
>> 
>> I think the main issue is the "indexer: relocation error" but I'm not 
>> completely sure. I see that the app uses enable_star which has been removed 
>> but I don't think this is what's causing the problem.
>> 
>> I'm guessing that I need to yum install a dependency that I'm missing but I 
>> don't know what, if anyone could point me in the right direction that would 
>> be very much appreciated!
>> 
>> Thanks!
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphi...@googlegroups.com <>.
>> To post to this group, send email to thinkin...@googlegroups.com <>.
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/think

Re: [ts] Sphinx error in passenger, works in console

[Pat Allan]

I’m still out of ideas for the moment.

But as for real-time indices, this is a good place to start:
https://freelancing-gods.com/thinking-sphinx/real_time.html 
<https://freelancing-gods.com/thinking-sphinx/real_time.html>

—
Pat

> On 12 Mar 2018, at 2:56 am, mrmanishs <mani...@acceleweb.com 
> <mailto:mani...@acceleweb.com>> wrote:
> 
> Thanks Pat for the follow-up.
> 
> Yes, was happening on each and every request. As I stated, it only happened 
> through Passenger also, not through console.
> 
> I think I'll switch to real-time indices, if you think of anything else, let 
> me know. It's got me stumped.
> 
> On Sunday, March 11, 2018 at 12:12:56 AM UTC-5, Pat Allan wrote:
> I know it’s hard to test now that you’ve disabled deltas, but I wondering… 
> did it happen on every single request?
> 
> It’s just that, previously - as already noted - failed indexing didn’t raise 
> exceptions. If two requests came at the same time that both made alterations 
> to the same model, one of the delta processing attempts would work, the other 
> wouldn’t - but the end result is still both alterations would be in Sphinx 
> (the successful processing captured both changes).
> 
> Now, in the same scenario, one would work, and the other would fail and raise 
> an exception. The data would still be persisted in Sphinx (and given deltas 
> are invoked in after_commit, in the database as well), but the 500 error 
> resulting from the failure is a terrible user experience.
> 
> If this is the situation, then the best solution is to either shift delta 
> processing into a background worker (via ts-sidekiq-delta, ts-delayed-delta, 
> or ts-resque-delta), or to use real-time indices.
> 
> Of course, if this isn’t the case - if a single request raises the exception, 
> and there’s no other concurrent requests altering the same model, then the 
> cause will be something else (and I’m really not sure what it could be!). 
> Perhaps real-time indices are worth investigating either way? They’ve been 
> around in TS for a good while now and I use them for any of my new projects 
> that need Sphinx, as do significantly sized Flying Sphinx customers.
> 
>> On 11 Mar 2018, at 2:39 pm, mrmanishs <man...@acceleweb.com <>> wrote:
>> 
>> Thanks for getting back Pat. Yes deltas were working great on the old server.
>> 
>> The permission is 644 and owner is root.
>> 
>> On Saturday, March 10, 2018 at 7:45:48 PM UTC-5, Pat Allan wrote:
>> So, one thing that’s probably changed in that upgrade is that the newer 
>> versions of Riddle are now a little fussier about running indexing commands, 
>> and raise exceptions depending on the status code of the `indexer` command. 
>> So, it could be possible this issue was always happening but the errors 
>> weren’t apparent. I feel that’s unlikely though - you’d probably have 
>> noticed if delta results weren’t appearing?
>> 
>> Also worth considering is the underlying error:
>> 
>>> FATAL: failed to lock 
>>> /sites/cap/classifieds.acceleweb.com/db/sphinx/production/address_delta.spl:
>>>  <http://classifieds.acceleweb.com/db/sphinx/production/address_delta.spl:> 
>>> Resource temporarily unavailable, will not index
>> 
>> I presume the path there is correct? What are the permissions on that file? 
>> Is the owner root?
>> 
>> —
>> Pat
>> 
>>> On 10 Mar 2018, at 1:12 am, mrmanishs <man...@acceleweb.com <>> wrote:
>>> 
>>> Thanks. I actually migrated to a new server and installed the latest 
>>> version of Sphinx/TS.
>>> 
>>> New server: Sphinx 3.0.2, TS 3.4.2 on Rails 5.1.5
>>> Old server: Sphinx 2.2.6, TS 3.2.0 on Rails 4.1.6
>>> 
>>> Could I have missed something in the upgrade that these symptoms may show?
>>> 
>>> I have disabled deltas for now to get it going, but it's not ideal.
>>> 
>>> 
>>> On Friday, March 9, 2018 at 8:58:02 AM UTC-5, Pat Allan wrote:
>>> Hmm. Have you tried running a rebuild? And has this problem started only 
>>> recently?
>>> 
>>> As a stop-gap solution, you could disable deltas and run the `ts:index` 
>>> task more regularly, but granted, that’s not ideal.
>>> 
>>>> On 9 Mar 2018, at 2:36 pm, mrmanishs <man...@ <>acceleweb.com 
>>>> <http://acceleweb.com/>> wrote:
>>>> 
>>>> 
>>>> I have nginx in front of passenger, so all the passenger log entries go 
>>>> into the nginx one. The only item in nginx is the url calls come back with 
>>>> a 500.
>>>> 

Re: [ts] Sphinx error in passenger, works in console

[Pat Allan]

I know it’s hard to test now that you’ve disabled deltas, but I wondering… did 
it happen on every single request?

It’s just that, previously - as already noted - failed indexing didn’t raise 
exceptions. If two requests came at the same time that both made alterations to 
the same model, one of the delta processing attempts would work, the other 
wouldn’t - but the end result is still both alterations would be in Sphinx (the 
successful processing captured both changes).

Now, in the same scenario, one would work, and the other would fail and raise 
an exception. The data would still be persisted in Sphinx (and given deltas are 
invoked in after_commit, in the database as well), but the 500 error resulting 
from the failure is a terrible user experience.

If this is the situation, then the best solution is to either shift delta 
processing into a background worker (via ts-sidekiq-delta, ts-delayed-delta, or 
ts-resque-delta), or to use real-time indices.

Of course, if this isn’t the case - if a single request raises the exception, 
and there’s no other concurrent requests altering the same model, then the 
cause will be something else (and I’m really not sure what it could be!). 
Perhaps real-time indices are worth investigating either way? They’ve been 
around in TS for a good while now and I use them for any of my new projects 
that need Sphinx, as do significantly sized Flying Sphinx customers.

> On 11 Mar 2018, at 2:39 pm, mrmanishs <mani...@acceleweb.com 
> <mailto:mani...@acceleweb.com>> wrote:
> 
> Thanks for getting back Pat. Yes deltas were working great on the old server.
> 
> The permission is 644 and owner is root.
> 
> On Saturday, March 10, 2018 at 7:45:48 PM UTC-5, Pat Allan wrote:
> So, one thing that’s probably changed in that upgrade is that the newer 
> versions of Riddle are now a little fussier about running indexing commands, 
> and raise exceptions depending on the status code of the `indexer` command. 
> So, it could be possible this issue was always happening but the errors 
> weren’t apparent. I feel that’s unlikely though - you’d probably have noticed 
> if delta results weren’t appearing?
> 
> Also worth considering is the underlying error:
> 
>> FATAL: failed to lock 
>> /sites/cap/classifieds.acceleweb.com/db/sphinx/production/address_delta.spl: 
>> <http://classifieds.acceleweb.com/db/sphinx/production/address_delta.spl:> 
>> Resource temporarily unavailable, will not index
> 
> I presume the path there is correct? What are the permissions on that file? 
> Is the owner root?
> 
> —
> Pat
> 
>> On 10 Mar 2018, at 1:12 am, mrmanishs <man...@acceleweb.com <>> wrote:
>> 
>> Thanks. I actually migrated to a new server and installed the latest version 
>> of Sphinx/TS.
>> 
>> New server: Sphinx 3.0.2, TS 3.4.2 on Rails 5.1.5
>> Old server: Sphinx 2.2.6, TS 3.2.0 on Rails 4.1.6
>> 
>> Could I have missed something in the upgrade that these symptoms may show?
>> 
>> I have disabled deltas for now to get it going, but it's not ideal.
>> 
>> 
>> On Friday, March 9, 2018 at 8:58:02 AM UTC-5, Pat Allan wrote:
>> Hmm. Have you tried running a rebuild? And has this problem started only 
>> recently?
>> 
>> As a stop-gap solution, you could disable deltas and run the `ts:index` task 
>> more regularly, but granted, that’s not ideal.
>> 
>>> On 9 Mar 2018, at 2:36 pm, mrmanishs <man...@ <>acceleweb.com 
>>> <http://acceleweb.com/>> wrote:
>>> 
>>> 
>>> I have nginx in front of passenger, so all the passenger log entries go 
>>> into the nginx one. The only item in nginx is the url calls come back with 
>>> a 500.
>>> 
>>> Thanks for the note.
>>> 
>>> Any other ideas? Also, up for doing some real-time debugging if it would 
>>> help. I am not really sure what's going on and it's impacting functionality 
>>> on site :(
>>> 
>>> 
>>> 
>>> On Thursday, March 8, 2018 at 5:14:47 PM UTC-5, Pat Allan wrote:
>>> Hmm. Is there anything related to this issue in your passenger logs? And/or 
>>> if you’re using nginx/apache/etc in front of that, their logs?
>>> 
>>> Unrelated, but: indexed_models is no longer needed/used by TS v3+.
>>> 
>>> —
>>> Pat
>>> 
>>>> On 9 Mar 2018, at 1:04 am, mrmanishs <man...@acceleweb.com <>> wrote:
>>>> 
>>>> Thanks Pat.
>>>> 
>>>> - searchd and nginx are both being run as root. Everything's root 
>>>> actually...
>>>> - I have searchd in /usr/local/bin and confirmed the production bin_path

Re: [ts] Sphinx error in passenger, works in console

[Pat Allan]

So, one thing that’s probably changed in that upgrade is that the newer 
versions of Riddle are now a little fussier about running indexing commands, 
and raise exceptions depending on the status code of the `indexer` command. So, 
it could be possible this issue was always happening but the errors weren’t 
apparent. I feel that’s unlikely though - you’d probably have noticed if delta 
results weren’t appearing?

Also worth considering is the underlying error:

> FATAL: failed to lock 
> /sites/cap/classifieds.acceleweb.com/db/sphinx/production/address_delta.spl: 
> <http://classifieds.acceleweb.com/db/sphinx/production/address_delta.spl:> 
> Resource temporarily unavailable, will not index

I presume the path there is correct? What are the permissions on that file? Is 
the owner root?

—
Pat

> On 10 Mar 2018, at 1:12 am, mrmanishs <mani...@acceleweb.com 
> <mailto:mani...@acceleweb.com>> wrote:
> 
> Thanks. I actually migrated to a new server and installed the latest version 
> of Sphinx/TS.
> 
> New server: Sphinx 3.0.2, TS 3.4.2 on Rails 5.1.5
> Old server: Sphinx 2.2.6, TS 3.2.0 on Rails 4.1.6
> 
> Could I have missed something in the upgrade that these symptoms may show?
> 
> I have disabled deltas for now to get it going, but it's not ideal.
> 
> 
> On Friday, March 9, 2018 at 8:58:02 AM UTC-5, Pat Allan wrote:
> Hmm. Have you tried running a rebuild? And has this problem started only 
> recently?
> 
> As a stop-gap solution, you could disable deltas and run the `ts:index` task 
> more regularly, but granted, that’s not ideal.
> 
>> On 9 Mar 2018, at 2:36 pm, mrmanishs <man...@ <>acceleweb.com 
>> <http://acceleweb.com/>> wrote:
>> 
>> 
>> I have nginx in front of passenger, so all the passenger log entries go into 
>> the nginx one. The only item in nginx is the url calls come back with a 500.
>> 
>> Thanks for the note.
>> 
>> Any other ideas? Also, up for doing some real-time debugging if it would 
>> help. I am not really sure what's going on and it's impacting functionality 
>> on site :(
>> 
>> 
>> 
>> On Thursday, March 8, 2018 at 5:14:47 PM UTC-5, Pat Allan wrote:
>> Hmm. Is there anything related to this issue in your passenger logs? And/or 
>> if you’re using nginx/apache/etc in front of that, their logs?
>> 
>> Unrelated, but: indexed_models is no longer needed/used by TS v3+.
>> 
>> —
>> Pat
>> 
>>> On 9 Mar 2018, at 1:04 am, mrmanishs <man...@acceleweb.com <>> wrote:
>>> 
>>> Thanks Pat.
>>> 
>>> - searchd and nginx are both being run as root. Everything's root 
>>> actually...
>>> - I have searchd in /usr/local/bin and confirmed the production bin_path is 
>>> /usr/local/bin . I have this in my thinking_sphinx.yml file for prod:
>>> 
>>> production:
>>>   mysql41: 9401
>>>   configuration_file: ""
>>>   log: ""
>>>   query_log: ""
>>>   pid_file: ""
>>>   min_infix_len: 3
>>>   bin_path: '/usr/local/bin'
>>>   indexed_models:
>>> - Address
>>>   quiet_deltas: true
>>> 
>>> Any other thoughts?
>>> 
>>> On Wednesday, March 7, 2018 at 11:45:43 PM UTC-5, Pat Allan wrote:
>>> Hi Manish,
>>> 
>>> I’m at a conference today, so please excuse the brevity, but it sounds like 
>>> this might be a permissions issue. See this note in the documentation:
>>> https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas 
>>> <https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas>
>>> (Both the deltas section and possibly the following section for passenger)
>>> 
>>> Certainly, real-time indices shouldn’t hit these issues, but you should 
>>> certainly be able to get your current setup working.
>>> 
>>> Hope this helps!
>>> 
>>> —
>>> Pat
>>> 
>>>> On 8 Mar 2018, at 2:58 pm, mrmanishs <man...@acceleweb.com <>> wrote:
>>>> 
>>>> After further investigation, it looks like an issue with delta indexing. 
>>>> Is there a fix? Should real-time indices be used instead, is that a better 
>>>> approach?
>>>> 
>>>> On Wednesday, March 7, 2018 at 4:39:16 PM UTC-5, mrmanishs wrote:
>>>> Whenever I try to do anything in my app through passenger, I get an error 
>>>> similar to this:
>>>> 
>>>> F, [2018-03-07T21:35:13.949318 #30762] FATAL -- : 
>>>> [d6484d2

Re: [ts] Sphinx error in passenger, works in console

[Pat Allan]

Hmm. Have you tried running a rebuild? And has this problem started only 
recently?

As a stop-gap solution, you could disable deltas and run the `ts:index` task 
more regularly, but granted, that’s not ideal.

> On 9 Mar 2018, at 2:36 pm, mrmanishs <mani...@acceleweb.com 
> <mailto:mani...@acceleweb.com>> wrote:
> 
> 
> I have nginx in front of passenger, so all the passenger log entries go into 
> the nginx one. The only item in nginx is the url calls come back with a 500.
> 
> Thanks for the note.
> 
> Any other ideas? Also, up for doing some real-time debugging if it would 
> help. I am not really sure what's going on and it's impacting functionality 
> on site :(
> 
> 
> 
> On Thursday, March 8, 2018 at 5:14:47 PM UTC-5, Pat Allan wrote:
> Hmm. Is there anything related to this issue in your passenger logs? And/or 
> if you’re using nginx/apache/etc in front of that, their logs?
> 
> Unrelated, but: indexed_models is no longer needed/used by TS v3+.
> 
> —
> Pat
> 
>> On 9 Mar 2018, at 1:04 am, mrmanishs <man...@acceleweb.com <>> wrote:
>> 
>> Thanks Pat.
>> 
>> - searchd and nginx are both being run as root. Everything's root actually...
>> - I have searchd in /usr/local/bin and confirmed the production bin_path is 
>> /usr/local/bin . I have this in my thinking_sphinx.yml file for prod:
>> 
>> production:
>>   mysql41: 9401
>>   configuration_file: ""
>>   log: ""
>>   query_log: ""
>>   pid_file: ""
>>   min_infix_len: 3
>>   bin_path: '/usr/local/bin'
>>   indexed_models:
>> - Address
>>   quiet_deltas: true
>> 
>> Any other thoughts?
>> 
>> On Wednesday, March 7, 2018 at 11:45:43 PM UTC-5, Pat Allan wrote:
>> Hi Manish,
>> 
>> I’m at a conference today, so please excuse the brevity, but it sounds like 
>> this might be a permissions issue. See this note in the documentation:
>> https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas 
>> <https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas>
>> (Both the deltas section and possibly the following section for passenger)
>> 
>> Certainly, real-time indices shouldn’t hit these issues, but you should 
>> certainly be able to get your current setup working.
>> 
>> Hope this helps!
>> 
>> —
>> Pat
>> 
>>> On 8 Mar 2018, at 2:58 pm, mrmanishs <man...@acceleweb.com <>> wrote:
>>> 
>>> After further investigation, it looks like an issue with delta indexing. Is 
>>> there a fix? Should real-time indices be used instead, is that a better 
>>> approach?
>>> 
>>> On Wednesday, March 7, 2018 at 4:39:16 PM UTC-5, mrmanishs wrote:
>>> Whenever I try to do anything in my app through passenger, I get an error 
>>> similar to this:
>>> 
>>> F, [2018-03-07T21:35:13.949318 #30762] FATAL -- : 
>>> [d6484d23-620d-4f33-8f17-13c12b7dab53] Riddle::CommandFailedError (Sphinx 
>>> command failed to execute):
>>> F, [2018-03-07T21:35:13.949426 #30762] FATAL -- : 
>>> [d6484d23-620d-4f33-8f17-13c12b7dab53]
>>> F, [2018-03-07T21:35:13.949536 #30762] FATAL -- : 
>>> [d6484d23-620d-4f33-8f17-13c12b7dab53] 
>>> app/controllers/listing_controller.rb:
>>> 
>>> 
>>> That's all I get... no other information.
>>> 
>>> The same instructions work in console and there are no issues. Everything 
>>> else on the rails app works fine.
>>> 
>>> Any idea? What else can I do to get more information?
>>> 
>>> Manish
>>> 
>>> 
>>> 
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "Thinking Sphinx" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to thinking-sphi...@googlegroups.com <>.
>>> To post to this group, send email to thinkin...@googlegroups.com <>.
>>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>>> <https://groups.google.com/group/thinking-sphinx>.
>>> For more options, visit https://groups.google.com/d/optout 
>>> <https://groups.google.com/d/optout>.
>> 
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphi...@googlegroups.com

Re: [ts] Sphinx error in passenger, works in console

[Pat Allan]

Hmm. Is there anything related to this issue in your passenger logs? And/or if 
you’re using nginx/apache/etc in front of that, their logs?

Unrelated, but: indexed_models is no longer needed/used by TS v3+.

—
Pat

> On 9 Mar 2018, at 1:04 am, mrmanishs <mani...@acceleweb.com 
> <mailto:mani...@acceleweb.com>> wrote:
> 
> Thanks Pat.
> 
> - searchd and nginx are both being run as root. Everything's root actually...
> - I have searchd in /usr/local/bin and confirmed the production bin_path is 
> /usr/local/bin . I have this in my thinking_sphinx.yml file for prod:
> 
> production:
>   mysql41: 9401
>   configuration_file: ""
>   log: ""
>   query_log: ""
>   pid_file: ""
>   min_infix_len: 3
>   bin_path: '/usr/local/bin'
>   indexed_models:
> - Address
>   quiet_deltas: true
> 
> Any other thoughts?
> 
> On Wednesday, March 7, 2018 at 11:45:43 PM UTC-5, Pat Allan wrote:
> Hi Manish,
> 
> I’m at a conference today, so please excuse the brevity, but it sounds like 
> this might be a permissions issue. See this note in the documentation:
> https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas 
> <https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas>
> (Both the deltas section and possibly the following section for passenger)
> 
> Certainly, real-time indices shouldn’t hit these issues, but you should 
> certainly be able to get your current setup working.
> 
> Hope this helps!
> 
> —
> Pat
> 
>> On 8 Mar 2018, at 2:58 pm, mrmanishs <man...@acceleweb.com <>> wrote:
>> 
>> After further investigation, it looks like an issue with delta indexing. Is 
>> there a fix? Should real-time indices be used instead, is that a better 
>> approach?
>> 
>> On Wednesday, March 7, 2018 at 4:39:16 PM UTC-5, mrmanishs wrote:
>> Whenever I try to do anything in my app through passenger, I get an error 
>> similar to this:
>> 
>> F, [2018-03-07T21:35:13.949318 #30762] FATAL -- : 
>> [d6484d23-620d-4f33-8f17-13c12b7dab53] Riddle::CommandFailedError (Sphinx 
>> command failed to execute):
>> F, [2018-03-07T21:35:13.949426 #30762] FATAL -- : 
>> [d6484d23-620d-4f33-8f17-13c12b7dab53]
>> F, [2018-03-07T21:35:13.949536 #30762] FATAL -- : 
>> [d6484d23-620d-4f33-8f17-13c12b7dab53] app/controllers/listing_controller.rb:
>> 
>> 
>> That's all I get... no other information.
>> 
>> The same instructions work in console and there are no issues. Everything 
>> else on the rails app works fine.
>> 
>> Any idea? What else can I do to get more information?
>> 
>> Manish
>> 
>> 
>> 
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphi...@googlegroups.com <>.
>> To post to this group, send email to thinkin...@googlegroups.com <>.
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] Sphinx error in passenger, works in console

[Pat Allan]

Hi Manish,

I’m at a conference today, so please excuse the brevity, but it sounds like 
this might be a permissions issue. See this note in the documentation:
https://freelancing-gods.com/thinking-sphinx/common_issues.html#deltas 

(Both the deltas section and possibly the following section for passenger)

Certainly, real-time indices shouldn’t hit these issues, but you should 
certainly be able to get your current setup working.

Hope this helps!

—
Pat

> On 8 Mar 2018, at 2:58 pm, mrmanishs  > wrote:
> 
> After further investigation, it looks like an issue with delta indexing. Is 
> there a fix? Should real-time indices be used instead, is that a better 
> approach?
> 
> On Wednesday, March 7, 2018 at 4:39:16 PM UTC-5, mrmanishs wrote:
> Whenever I try to do anything in my app through passenger, I get an error 
> similar to this:
> 
> F, [2018-03-07T21:35:13.949318 #30762] FATAL -- : 
> [d6484d23-620d-4f33-8f17-13c12b7dab53] Riddle::CommandFailedError (Sphinx 
> command failed to execute):
> F, [2018-03-07T21:35:13.949426 #30762] FATAL -- : 
> [d6484d23-620d-4f33-8f17-13c12b7dab53]
> F, [2018-03-07T21:35:13.949536 #30762] FATAL -- : 
> [d6484d23-620d-4f33-8f17-13c12b7dab53] app/controllers/listing_controller.rb:
> 
> 
> That's all I get... no other information.
> 
> The same instructions work in console and there are no issues. Everything 
> else on the rails app works fine.
> 
> Any idea? What else can I do to get more information?
> 
> Manish
> 
> 
> 
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] Sphinx on Elastic Beanstalk

[Pat Allan]

Hi Evan,

I think you’re right about it being a dependency issue. Sphinx will require 
mysql-related dependencies - you don’t need a MySQL server running, but you 
will need client libraries installed. I’m not across yum, so I’m not sure 
what’s ideal. Also, it’s worth noting that you may need to reinstall Sphinx 
after the mysql dependencies, to ensure it picks up on them correctly.

Do let us know if you find a solution!

—
Pat

> On 4 Jan 2018, at 8:29 am, Evan Shabsove  > wrote:
> 
> Hey everyone,
> 
> I've been trying to set up sphinx on an elastic beanstalk instance. I 
> currently have searchd running, but when I try to run the command
> 
> RAILS_ENV=production bundle exec rake ts:index
> 
> I get an error of
> 
> using config file '/var/app/ondeck/config/production.sphinx.conf'...
> WARNING: key 'enable_star' was permanently removed from Sphinx configuration. 
> Refer to documentation for details.
> WARNING: key 'enable_star' was permanently removed from Sphinx configuration. 
> Refer to documentation for details.
> indexing index 'custom_field_value_core'...
> indexer: relocation error: indexer: symbol mysql_init, version 
> libmysqlclient_18 not defined in file libmysqlclient.so.18 with link time 
> reference
> 
> When I try to make a search query on the website I get an error of
> 
> "free":"no enabled local indexes to search
> 
> 
> I think the main issue is the "indexer: relocation error" but I'm not 
> completely sure. I see that the app uses enable_star which has been removed 
> but I don't think this is what's causing the problem.
> 
> I'm guessing that I need to yum install a dependency that I'm missing but I 
> don't know what, if anyone could point me in the right direction that would 
> be very much appreciated!
> 
> Thanks!
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] Updating indexes after failed generation

[Pat Allan]

Hey :)

Running the regenerate command does a bunch of things:

* stops the daemon
* deletes the existing index files
* reconfigures Sphinx
* starts the daemon
* populates the index data (e.g. the generate task)

Given the process failed for you on the last step, you can just run the 
generate task, avoiding further downtime of the daemon. Also, given you’re 
using real-time indices, that’ll always be invoked with Flying Sphinx using the 
Thinking Sphinx rake task: `heroku run rake ts:generate`.

As for processing single indices, this feature was added in v3.4.0, and is done 
via an environment variable:

INDEX_FILTER=article_core rake ts:generate

But, on the topic of upgrading to v3.4.x - the rake tasks have now been 
unified, and I’ve opted for the original set of task names (historically, for 
SQL indices) to perform all the relevant behaviour for both SQL and real-time 
indices. In development, this means you can just use `ts:rebuild` instead of 
`ts:regenerate`, and `ts:index` instead of `ts:generate`. When it comes to 
mixing with Heroku, then right now it’s probably best using the `flying-sphinx 
regenerate` command and `ts:rt:index` task respectively.

That said, the original real-time tasks (regenerate and generate) still exist, 
but are deprecated.

I’ll look into improving the Flying Sphinx docs and integration to make this a 
little more obvious!

—
Pat

> On 28 Nov 2017, at 10:59 am, Francisco Trindade  > wrote:
> 
> Hi all,
> 
> 
> we currently run a daily task to process our indices (heroku run bundle exec 
> flying-sphinx index).
> 
> 
> Since we have changed a index, last night we had to run a regenerate task to 
> generate them again (heroku run bundle exec flying-sphinx regenerate). 
> However, midway through the regeneration the process failed because one of 
> the objects being indexed had invalid data, which means one of our indices is 
> not complete.
> 
> My question is if I need to regenerate the indices again since the last task 
> didn't complete, or if we just process the index that failed and that will be 
> fine. If regenerate is necessary, is there a way to regenerate a single index 
> without stopping the sphinx server?
> 
> Thanks
> Francisco
> 
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] Records for has many association disappearing

[Pat Allan]

I’m not sure how all of this will impact the file ownership, but are you able 
to check via an SSH connection who the owner of the index files are after a 
rebuild, and after a delta index is processed?

> On 8 Nov 2017, at 2:46 am, mikej  > wrote:
> 
> Right, perhaps this is worth looking into.
> 
> The application is on a managed server and the hosting company set up cron 
> jobs and monitoring.  I use systemd to index/rebuild.  This will therefore 
> end up with the application user’s service manager user.  I also note that 
> cron jobs (index and rebuild) are set up for the application user but use 
> $HOME/bin/reindex and $HOME/bin/rebuild.  Does this help?
> 
> Sincere thanks,
> 
> Mike
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] Records for has many association disappearing

[Pat Allan]

Hi Mike,

Mostly I’m really not sure what the cause is. But my next point of 
investigation would be to confirm the system user of the files, and whether 
that’s always consistent between indexing, rebuilding, and delta indexing - is 
it always the same user in each of those cases?

—
Pat

> On 3 Nov 2017, at 5:30 am, mikej  > wrote:
> 
> I have been looking into this a little more, so hoping the shed a bit more 
> light on it.
> 
> Note:
> There are no clues in nginx logs
> An index fixes the problem, in that you can see the missing records.
> 
> I have a function that updates some client_orders for a client.  For a while 
> any updated records were disappearing.  When I ran rebuild or index the 
> records appeared again but if I updated the records they would disappear.  
> Only when I ran:
> 
> rm -r db/sphinx;rm config/production.sphinx.conf;rm -r tmp/binlog/production;
> then rebuild
> 
> could I repeat the task several times without records disappearing.  There 
> are no callbacks.
> 
> Any help?
> 
> Huge thanks,
> 
> Mike
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [ts] Re: Records for has many association disappearing

[Pat Allan]

One more thing to check that I’ve just thought of:

>From memory, errors related to inline deltas (as opposed to ones processed by 
>a background worker such as Sidekiq or DelayedJob) can end up in the web 
>server logs (puma/passenger/etc), so that could be another place to look for 
>clues.

— 
Pat

> On 5 Oct 2017, at 10:50 pm, mikej  > wrote:
> 
> bin_path is already set and looks correct.  I'll try ts:index next time it 
> happens and see if that fixes it.  If so, I reckon I'll set that as a cron 
> job so the problem should fix itself.  Not ideal, but sounds like the best we 
> can do.  
> 
> Thanks again,
> 
> Mike
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Re: Records for has many association disappearing

[Pat Allan]

Thanks for those details Mike.

At this point, my main suspicion is focused on the delta indexing.

Generally, the behaviour is as follows:

A ClientOrder instance is updated.
The delta processing fires, which has two parts:
Mark the core index records for that ClientOrder as deleted.
Process the ClientOrder delta index
Further searches find the ClientOrder instance from the delta index, ignoring 
the marked-as-deleted details in the core index.
When ts:index or ts:rebuild is fired, the changes available in the delta are 
now present in the core (and no longer marked as deleted).

I wonder if the "Process the ClientOrder delta index” step isn’t working - 
thus, the core records are marked as deleted, but there’s nothing new in the 
delta index, and so the updated instances aren’t present in searches. In your 
setup, this can happen when the web processes where the updating is happening 
don’t have knowledge of where Sphinx’s indexer tool is located - they don’t 
have the same PATH setup as your cron job or shell. The fix for this is to set 
bin_path in your config/thinking_sphinx.yml file for the appropriate 
environments:
https://freelancing-gods.com/thinking-sphinx/common_issues.html#passenger 


If you’ve already got bin_path set - and it’s the correct path - then I’m 
really not sure what is the cause of the problem.

As an aside, and I don’t think this is part of the problem, but I’d recommend 
just using ts:index instead of ts:rebuild - there shouldn’t be any need to stop 
Sphinx, delete all the files, and recreate the setup from scratch. At the very 
least, it’d be useful to know if ts:index does resolve the issue you’re seeing 
if the bin_path option doesn’t lead anywhere.

Cheers,

— 
Pat

> On 5 Oct 2017, at 9:21 pm, mikej  > wrote:
> 
> Hi Pat,
> 
> I can't honestly remember if 'rake ts:index' fixes it.  I think not which is 
> why I went on to ts:rebuild.  I'll try again next time it happens.
> It does not happen every time I update ClientOrder.
> There are no callbacks impacting Client or Order.
> It just affects Sphinx index for ClientOrder.  There is no index for the 
> Order model although I did miss indexes in ClientOrder for Order as an 
> association. 
> E.g.
> indexes order.name,:as => :order_name
> Sphinx is hosted on the same machine as my app.
> I have never found any clues in the rails logs.
> 
> Thanks for you time, much appreciated.
> 
> Mike
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Records for has many association disappearing

[Pat Allan]

Hi Mike,

I’ve got some questions that may help us figure out the cause.

First, just to confirm: does a normal `rake ts:index` not get the data back 
into shape? And does this happen every time you update a ClientOrder instance?

Also, can you confirm whether there are any callbacks in ClientOrder that 
impact either Client or Order? And are the records missing in just the Sphinx 
index for ClientOrder? Or is it impacting Sphinx records for Order (if there’s 
an index on that model)?

And finally: you’re using the default delta approach - is Sphinx hosted on the 
same machine as your app? Is there anything in the Rails logs when you update a 
ClientOrder instance that provides clues to any problems?

Cheers,

— 
Pat

> On 5 Oct 2017, at 7:45 pm, mikej  > wrote:
> 
> I am having a problem with records disappearing.  It happens intermittently, 
> I can’t recreate it.  Any thoughts on how to fix or debug this would be much 
> appreciated.
> 
> Here is a simplified version of my situation:
> 
> class Client < ActiveRecord::Base
>   has_many :client_orders
>   has_many :orders, :through => : client_orders
> end
> 
> ThinkingSphinx::Index.define : client_order, :with => :active_record, :delta 
> => true do
>   indexes this and that
>   has client_id
> end
> 
> thinking-sphinx 3.2.0
> Sphinx 2.2.9
> 
> When a client_order is updated all the orders for that client disappear.  
> Orders for other clients are fine.  Rightly or wrongly I can fix it with:
> 
> rm -r db/sphinx;rm config/production.sphinx.conf;rm -r tmp/binlog/production;
> rebuild index
> 
> I have a cron job in place to rebuild indexes daily.
> 
> Any thoughts?
> 
> Many thanks,
> 
> Mike
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Could not log "warn.thinking_sphinx" event.

[Pat Allan]

Hi Koen,

It turns out this is two bugs, one of which I fixed a couple of days ago:
https://github.com/pat/thinking-sphinx/commit/60df63108b6a67b9740f8b277886526d8aa79738
 

And the other just now, due to your email:
https://github.com/pat/thinking-sphinx/commit/f12073f7f8a5f41ec6de536361c5ee2f2297b987
 


If you want, you can use the latest from the develop branch to get these fixes 
into place, but otherwise I should have 3.4.2 released in the next few days if 
all goes to plan.

Thanks for the help!

— 
Pat

> On 27 Sep 2017, at 12:00 am, Koen Eijsvogels  > wrote:
> 
> Hi,
> 
> In one of my projects, I suddenly got a strange error when trying to search 
> with a different ranker. 
> 
> To reproduce the problem, I created an empty Rails (5.1.4) project with only 
> one model (city):
> 
> class CreateCities < ActiveRecord::Migration[5.1]
>   def change
> create_table :cities do |t|
>   t.string :names
>   t.timestamps
> end
>   end
> end
> 
> and an index:
> 
> ThinkingSphinx::Index.define :city, :with => :active_record do
>   # fields
>   indexes names
> end
> 
> 
> After I have generated the Sphinx config file and I indexed the records I am 
> able to search e.g. City.searc('amsterdam')
> 
> But as soon as I add a ranking mode like: City.search('amsterdam', ranker: 
> :bm25) I get the warning below:
> 
> Could not log "warn.thinking_sphinx" event. NoMethodError: undefined method 
> `payload' for "  \e[1m\e[32mSphinx\e[0m  ":String 
> ["/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/active_record/log_subscriber.rb:19:in
>  `warn'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/active_record/log_subscriber.rb:19:in
>  `warn'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/subscriber.rb:99:in
>  `finish'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/log_subscriber.rb:83:in
>  `finish'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/notifications/fanout.rb:102:in
>  `finish'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/notifications/fanout.rb:46:in
>  `block in finish'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/notifications/fanout.rb:46:in
>  `each'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/notifications/fanout.rb:46:in
>  `finish'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/notifications/instrumenter.rb:42:in
>  `finish_with_state'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/notifications/instrumenter.rb:27:in
>  `instrument'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in
>  `instrument'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/logger.rb:3:in
>  `log'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/middlewares/valid_options.rb:16:in
>  `check_options'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/middlewares/valid_options.rb:5:in
>  `block in call'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/middlewares/valid_options.rb:5:in
>  `each'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/middlewares/valid_options.rb:5:in
>  `call'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/middlewares/stale_id_filter.rb:10:in
>  `call'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/middleware-0.1.0/lib/middleware/runner.rb:31:in
>  `call'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/middleware-0.1.0/lib/middleware/builder.rb:102:in
>  `call'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/search.rb:88:in
>  `populate'", 
> "/Users/koen/.rvm/gems/ruby-2.4.1@myproject/gems/thinking-sphinx-3.4.1/lib/thinking_sphinx/search.rb:138:in
>  `method_missing'", 
> "/Users/koen/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/irb/inspector.rb:109:in 
> `block in '", 
> "/Users/koen/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/irb/inspector.rb:102:in 
> `inspect_value'", 
> "/Users/koen/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/irb/context.rb:385:in 
> `inspect_last_value'", 
> "/Users/koen/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/irb.rb:665:in 
> `output_value'", 
> "/Users/koen/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/irb.rb:494:in 

Re: [ts] How to do full reindex outside of production machine

[Pat Allan]

Hi Loganathan,

Seven hours is indeed a long time for indexing! Here’s some related thoughts 
that may help you find a decent solution:

Optimise SQL statements
Given the amount of time involved, I’m curious as to how many records you may 
be indexing, and how complex the underlying SQL queries are. Have you confirmed 
you have database indices on foreign key columns?

Also, (and I’ve just noticed it’s not covered in the documentation) it’s 
possible to shift multi-value attributes into separate queries, which may make 
the overall speed faster (especially if they are creating joins that are only 
necessary for those attributes and not the rest of the index):

# This is only applicable for multi-value attributes:
has taggings.tag_id, :as => :tag_ids, :source => :query

# Also, you can use ranged queries, which may perform
# better depending on the data situation:
has taggings.tag_id, :as => :tag_ids, :source => :ranged_query

But yeah, I generally recommend streamlining the SQL queries as much as 
possible - it reduces the load on your database and speeds up indexing time.

Separate hosts for the database and Sphinx
Speaking of reducing load - do you have Sphinx and your database on the same 
server? If so, is having separate servers an option? This may slow down 
indexing slightly (as you’ve got the extra time for data transferring from one 
machine to the other) but should reduce the load on the database server, which 
is a good thing.

Setting the memory limit for the indexer
http://sphinxsearch.com/docs/current.html#conf-mem-limit 


Sphinx has the ability to cap the amount of memory used by the indexer. This 
may help limit the resources it uses and thus allow you to SSH into the machine 
while it’s running? You can set this per-environment in 
config/thinking_sphinx.yml.

Indexing data separately
This is what you’ve directly asked about. You can certainly run the indexer 
elsewhere and then copy it to the main Sphinx server, but there’s some things 
to note: Firstly, you will still want to have Sphinx reference the same 
production database.

And then, when copying, I would recommend you append .new. into the index file 
names (e.g. article_core.spa becomes article_core.new.spa), then copy the files 
over to your main machine, then send a SIGHUP signal to the daemon. It’ll then 
load the new index data in and replace the old files itself. If you’re running 
the indexer manually, then you can get the correct file names by running it 
with the --rotate flag - but otherwise: all Sphinx index files need to be 
renamed (not just spa files).

And yes, you may end up with slightly out-of-date data, but that’s going to be 
the case no matter what - especially due to the long time indexing requires.

I really hope there are possibilities for SQL optimisation, because this can 
have such a significant impact on indexing times. Otherwise, I hope the rest of 
the information is useful too.

Cheers,

— 
Pat

> On 7 Sep 2017, at 11:10 am, Loganathan Sellappa  > wrote:
> 
> Hi All,
> 
> Whenever we do the full re-indexing on our database, it is taking around 7 
> hours to complete due to large data sets. Even sometimes it is making our 
> server to go unresponsive and we couldn't able to do the SSH until the 
> indexer completes. 
> 
> I'm looking out for the way to run the indexer outside of our production 
> machine and copy the indexed file to our machine afterward. Just wanted to 
> know whether it'll cause any impact or not and I need clarifications for 
> below question
> 
> During the process of Indexing & copying to the production server, there will 
> be some delta difference will be there between my index file & production 
> server's index file, will it be handled automatically by the indexer or do I 
> need to stop the indexer on production server during the process?
> 
> 
> regards,
> Loganathan
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at 

Re: [ts] Real-time Indices and namespaced models

[Pat Allan]

Hi Koen,

Thanks for bringing this to my attention.

I’ve pushed a patch to the develop branch which automatically converts that 
callback argument to a symbol, and until that is deployed into a release, I've 
updated the docs to add the `.to_sym` in for the example of namespace callbacks.

Cheers,

— 
Pat

> On 2 Sep 2017, at 10:39 am, Koen Eijsvogels  > wrote:
> 
> Hi,
> 
> Yesterday I stumbled on a issue with a namespaced model which I wanted to 
> have real time indexed. 
> 
> The index files has a line like: ThinkingSphinx::Index.define 'vehicle/car', 
> :with => :real_time do
> 
> In the model class Vehicle::Car < ApplicationRecord I did put the callback: 
> after_save ThinkingSphinx::RealTime.callback_for('vehicle/car') 
> 
> However when saving records, the Sphinx Index wasn't updated. After searching 
> I found that internally Thinking Sphinx converts the index definition from a 
> string to a symbol, but the string in the callback is not converted.
> 
> Maybe it is a solution to cast the input in method callback_for in file 
> real_time.rb always to symbol?
> 
> Now I solved the issue by casting the string into a symbol in the after_save 
> definition itself:
> 
> after_save ThinkingSphinx::RealTime.callback_for('vehicle/car'.to_sym) 
> 
> If this is the preferred solution, maybe it can be mentioned on the page 
> where the real time indices are explained or on the page where namespaced 
> models are explained?
> 
> 
> 
> Best regards,
> 
> Koen
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] How to set infix & prefix indexing in sphinx search

[Pat Allan]

Hi Logan,

min_infix_len and min_prefix_len are index-specific settings, not 
field-specific. So, you should be able to set them in the index definition:

indexes title, :infixes => true
indexes notes, :prefixes => true

set_property :min_infix_len => 3, :min_prefix_len => 3

Does that do what you need it to?

— 
Pat

> On 30 Aug 2017, at 9:53 am, Loganathan Sellappa <loganathan...@gmail.com 
> <mailto:loganathan...@gmail.com>> wrote:
> 
> Hi Pat,
> 
> I'm getting the below warning message when I follow the steps provided by you.
> 
> WARNING: min_prefix_len=0, prefix_fields ignored
> WARNING: min_infix_len=0, infix_fields ignored
> 
> 
> FYI I want to enable infix indexing for one column of the table and would 
> like to use the prefix indexing for rest of the columns in the same table. 
> Copied below the code for your reference
> 
> ThinkingSphinx::Index.define(:message, DEFAULT_INDEX_OPTIONS) do
> 
>   indexes title, :infixes => true, min_infix_length: 3 # Need something 
> like this
>   indexes notes,, :prefixes => true, min_prefix_length: 3 # Need 
> something like this
> 
> 
>   has beetil.beetil_tags.id, :as => :tag_ids, :source => :query
>   has watches.user_id, :as => :watching_user_ids, :source => :query
> 
>   has "customer_viewable_at IS NOT NULL", :as => :customer_viewable, 
> :type => :boolean
> 
>has "0", :as => :assigned_user_id, :type => :integer
>has "0", :as => :closed, :type => :boolean
>has "0", :as => :requires_config_access, :type => :boolean
>has Searchable::BY_ALL_CUSTOMERS, :as => :customer_ids, :type => 
> :integer
>has "1", :as => :published, :type => :boolean
> end
> 
> Regards,
> Logan
> 
> On Tuesday, 29 August 2017 13:54:44 UTC+5:30, Pat Allan wrote:
> Hi Logan,
> 
> To do this, you can set it per field in the index definition:
> 
>   indexes name, :infixes => true
>   indexes location, :prefixes => true
> 
> The dict setting can only be set per-index, so if you want it for specific 
> indices, add this to those index definitions:
> 
>   set_property :dict => ‘keywords’
> 
> Or for all indices, add it to each environment in your 
> config/thinking_sphinx.yml file:
> 
>   development:
> dict: keywords
>   test:
> dict: keywords
>   production:
> dict: keywords
> 
> Though at some point, I believe keywords will be (if it isn’t already) the 
> default dict setting.
> 
> Cheers,
> 
> — 
> Pat
> 
>> On 29 Aug 2017, at 8:06 am, Loganathan Sellappa <logana...@ <>gmail.com 
>> <http://gmail.com/>> wrote:
>> 
>> Hello,
>> 
>> 
>> 
>> I want to have infix indexing for few fields and prefix indexing for some 
>> other fields in my table. 
>> 
>> Is there a way to do it in sphinx?
>> 
>> 
>> 
>> I'm looking for options infix_fields & prefix_fields in Sphinx with 
>> dict=keyword
>> 
>> 
>> 
>> Regards,
>> Logan
>> 
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphi...@ <>googlegroups.com <http://googlegroups.com/>.
>> To post to this group, send email to thinkin...@ <>googlegroups. 
>> <http://googlegroups.com/>com <http://googlegroups.com/>.
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] How to set infix & prefix indexing in sphinx search

[Pat Allan]

Hi Logan,

To do this, you can set it per field in the index definition:

  indexes name, :infixes => true
  indexes location, :prefixes => true

The dict setting can only be set per-index, so if you want it for specific 
indices, add this to those index definitions:

  set_property :dict => ‘keywords’

Or for all indices, add it to each environment in your 
config/thinking_sphinx.yml file:

  development:
dict: keywords
  test:
dict: keywords
  production:
dict: keywords

Though at some point, I believe keywords will be (if it isn’t already) the 
default dict setting.

Cheers,

— 
Pat

> On 29 Aug 2017, at 8:06 am, Loganathan Sellappa  > wrote:
> 
> Hello,
> 
> 
> 
> I want to have infix indexing for few fields and prefix indexing for some 
> other fields in my table. 
> 
> Is there a way to do it in sphinx?
> 
> 
> 
> I'm looking for options infix_fields & prefix_fields in Sphinx with 
> dict=keyword
> 
> 
> 
> Regards,
> Logan
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

[ts] v3.4.0

[Pat Allan]

Hi everyone

Given the time between releases, I’ve forgotten to post here about v3.3.0 (back 
in December), and now v3.4.0 is released. Release notes for both are available:
https://github.com/pat/thinking-sphinx/releases/tag/v3.3.0 

https://github.com/pat/thinking-sphinx/releases/tag/v3.4.0 


There are no breaking changes, though there are a few key features in v3.4.0 
which involve deprecating behaviour (hence, the next release in the future may 
have breaking changes). In particular, you’ll want to make sure you’re casting 
filter values to their appropriate types (rather than presuming strings will be 
automatically translated).

Any questions, just ask :)

Cheers

-- 
Pat

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Association has_many through returning strings for IDs

[Pat Allan]

Hi Andrew,

The syntax you’ve got looks fine to me. What I’d expect to happen in this 
situation is that the SQL query generated for populating the Sphinx index will 
take the certification_id values and concatenate them together with comma 
separators, and return that string value to Sphinx. Sphinx will pull it apart 
again into integers, it just expects a comma-delimited value for MVAs.

It sounds like this isn’t the behaviour you’re seeing, though. Can you share 
the sql_query value in the configuration file (config/development.sphinx.conf) 
for this index/source? And can you confirm that there’s sql_attr_multi = uint 
certification_ids from field in the source?

Thanks,

— 
Pat

> On 26 Jul 2017, at 2:58 am, Andrew Haust  > wrote:
> 
> I've been stuck on this for almost a full day now and have a read a bunch and 
> can't figure this out.
> 
> I'm in the midst up upgrading a rails 3 app to 4 (and eventually 5).  Right 
> now, I'm on rails 4.0.6 (before upgrading to 4.2).
> 
> My Sphinx version is 2.2.11 and my Thinking Sphinx version is 3.3.0.
> 
> I'm using postgres as the backend.
> 
> I'm on macOS Sierra and installed Sphinx with brew install sphinx 
> --with-mysql --with-postgresql.
> 
> Here is my model:
> 
> class Company < ActiveRecord::Base
>   # ..
>   has_many :company_certifications
>   has_many :certifications, through: :company_certifications
>   # ..
> end
> 
> ThinkingSphinx::Index.define :company, with: :active_record do
>   # ..
>   has company_certifications.certification_id, as: :certification_ids
>   # I've tried adding `multi: true` here as well as `type: :integer` and 
> combinations of both
>   # ..
> end
> 
> rake ts:rebuild works just fine but certification_ids is an array of strings 
> as opposed to integers!  This is, of course, causing issue with SELECT 
> statements in postgres.
> 
> Any help is greatly appreciated, of course.
> 
> Thank you!
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Configuration fails with error "undefined method '<<' for nil:NilClass"

[Pat Allan]

Hi Melody,

I think that error was due to your alternative id column being nil on creation? 
I’ve just pushed a commit that allows for that scenario, but if I’ve diagnosed 
correctly, I’d recommend ensuring the id is set before the initial save, just 
to be sure.

If I’ve got it wrong though, do let me know :)

— 
Pat

> On 20 Jul 2017, at 1:23 am, Melody B <mel...@sassafras.coop 
> <mailto:mel...@sassafras.coop>> wrote:
> 
> Similar-looking bug now here:
> 
> NoMethodError:
>   undefined method `*' for nil:NilClass
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-c45b52438001/lib/thinking_sphinx/core/index.rb:33:in
>  `document_id_for_key'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-c45b52438001/lib/thinking_sphinx/core/index.rb:29:in
>  `document_id_for_instance'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-c45b52438001/lib/thinking_sphinx/deltas/default_delta.rb:16:in
>  `delete'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-c45b52438001/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:14:in
>  `block in after_commit'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-c45b52438001/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:13:in
>  `each'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-c45b52438001/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:13:in
>  `after_commit'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-c45b52438001/lib/thinking_sphinx/callbacks.rb:7:in
>  `block (2 levels) in callbacks'
> 
> I'm guessing there will probably end up being a couple more of these? If I 
> find the time I can try to submit a PR, but I'm pretty swamped with other 
> commitments today. Thanks for being so responsive and helpful with this!
> 
> ~ Melody
> 
> On Wednesday, July 19, 2017 at 10:54:51 AM UTC-4, Pat Allan wrote:
> Okay, just pushed a commit to the develop branch which should help get the 
> ball rolling:
> https://github.com/pat/thinking-sphinx/commit/c45b5243800124f2c5d60f05fa501ed1cdca32d4
>  
> <https://github.com/pat/thinking-sphinx/commit/c45b5243800124f2c5d60f05fa501ed1cdca32d4>
> 
> Give that a shot, and if you hit any further problems, do let me know. PRs 
> are certainly welcome, though I’m well aware there’s a lot of code in TS, so 
> any confusion is completely understandable. :)
> 
> — 
> Pat
> 
>> On 20 Jul 2017, at 12:05 am, Pat Allan <p...@freelancing-gods.com 
>> > wrote:
>> 
>> Definitely looks like a bug to me - and now that I’m investigating further, 
>> it looks like there are several bugs. Am putting some integration specs in 
>> place and working on getting the alternative-id feature actually working! 
>> Though it’s midnight here, so it’s likely something I’ll get a bit further 
>> on tomorrow.
>> 
>>> On 19 Jul 2017, at 11:43 pm, Melody B <mel...@sassafras.coop > 
>>> wrote:
>>> 
>>> A little update:
>>> 
>>> I'm currently running the latest development version of thinking-sphinx 
>>> from github, riddle 2.2.0, and told sphinx to treat the uuid keys as 
>>> strings. Configuration generates now, but I'm getting the same error I was 
>>> when I locally patched my gem version before I reached out -- whenever I 
>>> try to save a new instance of my indexed model it fails with this stack 
>>> trace:
>>> 
>>> TypeError:
>>>   no implicit conversion of Fixnum into String
>>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/core/index.rb:29:in
>>>  `+'
>>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/core/index.rb:29:in
>>>  `document_id_for_key'
>>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/deltas/default_delta.rb:16:in
>>>  `delete'
>>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:14:in
>>>  `block in after_commit'
>>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:13:in
>>>  `each'
>>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:13:in
>>>  `after_commit'
>>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundle

Re: [ts] Configuration fails with error "undefined method '<<' for nil:NilClass"

[Pat Allan]

Okay, just pushed a commit to the develop branch which should help get the ball 
rolling:
https://github.com/pat/thinking-sphinx/commit/c45b5243800124f2c5d60f05fa501ed1cdca32d4
 
<https://github.com/pat/thinking-sphinx/commit/c45b5243800124f2c5d60f05fa501ed1cdca32d4>

Give that a shot, and if you hit any further problems, do let me know. PRs are 
certainly welcome, though I’m well aware there’s a lot of code in TS, so any 
confusion is completely understandable. :)

— 
Pat

> On 20 Jul 2017, at 12:05 am, Pat Allan <p...@freelancing-gods.com 
> <mailto:p...@freelancing-gods.com>> wrote:
> 
> Definitely looks like a bug to me - and now that I’m investigating further, 
> it looks like there are several bugs. Am putting some integration specs in 
> place and working on getting the alternative-id feature actually working! 
> Though it’s midnight here, so it’s likely something I’ll get a bit further on 
> tomorrow.
> 
>> On 19 Jul 2017, at 11:43 pm, Melody B <mel...@sassafras.coop 
>> <mailto:mel...@sassafras.coop>> wrote:
>> 
>> A little update:
>> 
>> I'm currently running the latest development version of thinking-sphinx from 
>> github, riddle 2.2.0, and told sphinx to treat the uuid keys as strings. 
>> Configuration generates now, but I'm getting the same error I was when I 
>> locally patched my gem version before I reached out -- whenever I try to 
>> save a new instance of my indexed model it fails with this stack trace:
>> 
>> TypeError:
>>   no implicit conversion of Fixnum into String
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/core/index.rb:29:in
>>  `+'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/core/index.rb:29:in
>>  `document_id_for_key'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/deltas/default_delta.rb:16:in
>>  `delete'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:14:in
>>  `block in after_commit'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:13:in
>>  `each'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:13:in
>>  `after_commit'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/callbacks.rb:7:in
>>  `block (2 levels) in callbacks'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:455:in
>>  `public_send'
>> 
>> Taking a cursory glance at the delete method in DefaultDelta I'm wondering 
>> if this is perhaps possibly the culprit?
>>  
>> def delete(index, instance)
>>   ThinkingSphinx::Deltas::DeleteJob.new(
>> index.name, index.document_id_for_key(instance.id)
>>   ).perform
>> end
>>  
>> I don't know much about how the pieces of this fit together, but I wouldn't 
>> expect instance.id to be the correct key to fetch on when an alternative 
>> primary key has been defined. I can continue to try to chase this down and 
>> perhaps submit a PR if you think this is likely a bug, or try to find a 
>> minimally reproducing example that I can share so that we can rule out 
>> configuration or environment issues? I'm open to suggestions on how best to 
>> proceed.
>> 
>> ~ Melody
>> 
>> On Wednesday, July 19, 2017 at 3:22:37 AM UTC-4, Pat Allan wrote:
>> It doesn’t look like you have MVAs - the example you shared is a field (via 
>> the `indexes` method), and your right, the joins using UUID columns to get 
>> to that data should be fine. The attributes (via the `has` method) you have 
>> listed, as you’ve noted, are UUIDs and strings should do the trick there.
>> 
>> So, maybe Sphinx and TS can actually do what’s needed? Do you want to talk 
>> about what’s now not working?
>> 
>> Cheers,
>> 
>> — 
>> Pat
>> 
>>> On 19 Jul 2017, at 12:49 pm, Melody B <mel...@sassafras.coop <>> wrote:
>>> 
>>> I'm not sure whether I need multi-value attributes or not, the only line I 
>>> have in here that's particularly similar to the one you quoted is this:
>>> 
>>> indexes choices.option.canonical_name, as: :option_names
>>> 
>>> In order to get that data, you would need 

Re: [ts] Configuration fails with error "undefined method '<<' for nil:NilClass"

[Pat Allan]

Definitely looks like a bug to me - and now that I’m investigating further, it 
looks like there are several bugs. Am putting some integration specs in place 
and working on getting the alternative-id feature actually working! Though it’s 
midnight here, so it’s likely something I’ll get a bit further on tomorrow.

> On 19 Jul 2017, at 11:43 pm, Melody B <mel...@sassafras.coop 
> <mailto:mel...@sassafras.coop>> wrote:
> 
> A little update:
> 
> I'm currently running the latest development version of thinking-sphinx from 
> github, riddle 2.2.0, and told sphinx to treat the uuid keys as strings. 
> Configuration generates now, but I'm getting the same error I was when I 
> locally patched my gem version before I reached out -- whenever I try to save 
> a new instance of my indexed model it fails with this stack trace:
> 
> TypeError:
>   no implicit conversion of Fixnum into String
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/core/index.rb:29:in
>  `+'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/core/index.rb:29:in
>  `document_id_for_key'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/deltas/default_delta.rb:16:in
>  `delete'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:14:in
>  `block in after_commit'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:13:in
>  `each'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/active_record/callbacks/delta_callbacks.rb:13:in
>  `after_commit'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/bundler/gems/thinking-sphinx-9860c9452279/lib/thinking_sphinx/callbacks.rb:7:in
>  `block (2 levels) in callbacks'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:455:in
>  `public_send'
> 
> Taking a cursory glance at the delete method in DefaultDelta I'm wondering if 
> this is perhaps possibly the culprit?
>  
> def delete(index, instance)
>   ThinkingSphinx::Deltas::DeleteJob.new(
> index.name, index.document_id_for_key(instance.id)
>   ).perform
> end
>  
> I don't know much about how the pieces of this fit together, but I wouldn't 
> expect instance.id to be the correct key to fetch on when an alternative 
> primary key has been defined. I can continue to try to chase this down and 
> perhaps submit a PR if you think this is likely a bug, or try to find a 
> minimally reproducing example that I can share so that we can rule out 
> configuration or environment issues? I'm open to suggestions on how best to 
> proceed.
> 
> ~ Melody
> 
> On Wednesday, July 19, 2017 at 3:22:37 AM UTC-4, Pat Allan wrote:
> It doesn’t look like you have MVAs - the example you shared is a field (via 
> the `indexes` method), and your right, the joins using UUID columns to get to 
> that data should be fine. The attributes (via the `has` method) you have 
> listed, as you’ve noted, are UUIDs and strings should do the trick there.
> 
> So, maybe Sphinx and TS can actually do what’s needed? Do you want to talk 
> about what’s now not working?
> 
> Cheers,
> 
> — 
> Pat
> 
>> On 19 Jul 2017, at 12:49 pm, Melody B <mel...@sassafras.coop <>> wrote:
>> 
>> I'm not sure whether I need multi-value attributes or not, the only line I 
>> have in here that's particularly similar to the one you quoted is this:
>> 
>> indexes choices.option.canonical_name, as: :option_names
>> 
>> In order to get that data, you would need to join choices and option on uuid 
>> columns, as all of these models have uuid primary & foreign keys. I don't 
>> think I have any that do this specifically for ids, I defined these to get 
>> it to stop crashing on configure:
>> 
>> has response_id, type: :string
>> has response.mission_id, type: :string
>> has questioning.question_id, type: :string
>> 
>> But it still doesn't seem to be working -- this may have more to do with my 
>> app code than thinking sphinx/sphinx though, I haven't gotten deep into 
>> sorting it out now that it's no longer completely broken. I'll try updating 
>> to develop, a newer riddle, and making sure I'm on a good version of sphinx 
>> (I'm on 2.2.11 locally, and could hopefully update production instances if 
>> necessary)
>> 
>> I've been getting slowly resigned to the fact that I may need to switch 
&g

Re: [ts] Configuration fails with error "undefined method '<<' for nil:NilClass"

[Pat Allan]

It doesn’t look like you have MVAs - the example you shared is a field (via the 
`indexes` method), and your right, the joins using UUID columns to get to that 
data should be fine. The attributes (via the `has` method) you have listed, as 
you’ve noted, are UUIDs and strings should do the trick there.

So, maybe Sphinx and TS can actually do what’s needed? Do you want to talk 
about what’s now not working?

Cheers,

— 
Pat

> On 19 Jul 2017, at 12:49 pm, Melody B <mel...@sassafras.coop 
> <mailto:mel...@sassafras.coop>> wrote:
> 
> I'm not sure whether I need multi-value attributes or not, the only line I 
> have in here that's particularly similar to the one you quoted is this:
> 
> indexes choices.option.canonical_name, as: :option_names
> 
> In order to get that data, you would need to join choices and option on uuid 
> columns, as all of these models have uuid primary & foreign keys. I don't 
> think I have any that do this specifically for ids, I defined these to get it 
> to stop crashing on configure:
> 
> has response_id, type: :string
> has response.mission_id, type: :string
> has questioning.question_id, type: :string
> 
> But it still doesn't seem to be working -- this may have more to do with my 
> app code than thinking sphinx/sphinx though, I haven't gotten deep into 
> sorting it out now that it's no longer completely broken. I'll try updating 
> to develop, a newer riddle, and making sure I'm on a good version of sphinx 
> (I'm on 2.2.11 locally, and could hopefully update production instances if 
> necessary)
> 
> I've been getting slowly resigned to the fact that I may need to switch 
> search solutions (and actually already tried switching to pg_search because 
> in terms of the actual search features we're taking advantage of our needs 
> are really very mild and removing an external dependeny seemed like a net 
> benefit in terms of future maintenance) but it can't do 
> highlighting/excerpting on searches against associations (weird bug) and 
> hasn't updated to expose an interface for postgres 9.6's exact phrase 
> matching yet. I've been trying to evaluate other solutions, but finding info 
> about which ones support the exact weird combination of constraints I'm 
> dealing with when they support each thing I want to do separately is a bit of 
> a challenge.
> 
> In any case, thanks so much for your help!
> 
> ~ Melody
> 
> On Tuesday, July 18, 2017 at 10:09:35 PM UTC-4, Pat Allan wrote:
> Hmm. The recent versions of Sphinx do support filtering on string attributes, 
> so you could possibly set the explicit `:type => :string` for attributes 
> based on UUID columns. However, that’ll only work for single-value attributes 
> (and additionally, you’ll want to use the latest in the `develop` branch for 
> TS and ensure the bundled version of the Riddle gem is v2.2.0, as there are 
> some changes around properly quoting string attributes in search queries).
> 
> If you’re not across the Sphinx terminology, a multi-value attribute (MVA) is 
> essentially an attribute with a data type of array, but the arrays can only 
> hold integers. So, in practice it might look something like the following in 
> a TS index definition:
> 
> has users.id <http://users.id/>, :as => :user_ids
> 
> I’d expect, however, that TS is fine with generating SQL joins across UUID 
> keys - it’s just the actual attribute values being the issue. If you need 
> multi-value UUID attributes, then it feels like your paths are either adding 
> fake foreign keys, or switching to a different search tool - and I’m sure 
> neither of those options are particularly simple. Fingers crossed that you 
> only need single-value attributes for the UUID columns?
> 
> Feel free to ask any further questions :)
> 
> — 
> Pat
> 
>> On 19 Jul 2017, at 11:36 am, Melody B <mel...@sassafras.coop <>> wrote:
>> 
>> Thanks, I forgot that I had locked my version to 3.1.x in my gemfile when I 
>> tried to update the gem, so I hadn't noticed that I wasn't on the latest. I 
>> feel a bit silly now. I'm now getting a different error relating to my 
>> foreign keys (which are also uuid) not mapping to a valid sphinx type. 
>> 
>> Is there a way around this besides adding additional fake foreign keys to go 
>> with my fake primary keys and putting those on every model I need sphinx to 
>> be aware of? Would there even be a way to configure thinking sphinx so that 
>> it tried to find the associated records through those instead? Unfortunately 
>> I genuinely do need UUID keys for this app.
>> 
>> ~ Melody
>> 
>> On Tuesday, July 18, 2017 at 9:11:04 PM UTC-4, Pat Allan wrote:
>> Hi Melody,
>> 
>>

Re: [ts] Configuration fails with error "undefined method '<<' for nil:NilClass"

[Pat Allan]

Hmm. The recent versions of Sphinx do support filtering on string attributes, 
so you could possibly set the explicit `:type => :string` for attributes based 
on UUID columns. However, that’ll only work for single-value attributes (and 
additionally, you’ll want to use the latest in the `develop` branch for TS and 
ensure the bundled version of the Riddle gem is v2.2.0, as there are some 
changes around properly quoting string attributes in search queries).

If you’re not across the Sphinx terminology, a multi-value attribute (MVA) is 
essentially an attribute with a data type of array, but the arrays can only 
hold integers. So, in practice it might look something like the following in a 
TS index definition:

has users.id, :as => :user_ids

I’d expect, however, that TS is fine with generating SQL joins across UUID keys 
- it’s just the actual attribute values being the issue. If you need 
multi-value UUID attributes, then it feels like your paths are either adding 
fake foreign keys, or switching to a different search tool - and I’m sure 
neither of those options are particularly simple. Fingers crossed that you only 
need single-value attributes for the UUID columns?

Feel free to ask any further questions :)

— 
Pat

> On 19 Jul 2017, at 11:36 am, Melody B <mel...@sassafras.coop 
> <mailto:mel...@sassafras.coop>> wrote:
> 
> Thanks, I forgot that I had locked my version to 3.1.x in my gemfile when I 
> tried to update the gem, so I hadn't noticed that I wasn't on the latest. I 
> feel a bit silly now. I'm now getting a different error relating to my 
> foreign keys (which are also uuid) not mapping to a valid sphinx type. 
> 
> Is there a way around this besides adding additional fake foreign keys to go 
> with my fake primary keys and putting those on every model I need sphinx to 
> be aware of? Would there even be a way to configure thinking sphinx so that 
> it tried to find the associated records through those instead? Unfortunately 
> I genuinely do need UUID keys for this app.
> 
> ~ Melody
> 
> On Tuesday, July 18, 2017 at 9:11:04 PM UTC-4, Pat Allan wrote:
> Hi Melody,
> 
> There was a couple of commits related to fixing alternative primary keys that 
> are part of v3.3.0 - I realise if you’re not familiar with this gem or Sphinx 
> it could be daunting, but I’d recommend upgrading if at all possible.
> 
> The release notes for v3.2.0 and v3.3.0 (the only releases since v3.1.4) 
> cover off notable changes - I’d say there’s a good chance you won’t hit any 
> issues though:
> https://github.com/pat/thinking-sphinx/releases/tag/v3.2.0 
> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fpat%2Fthinking-sphinx%2Freleases%2Ftag%2Fv3.2.0=D=1=AFQjCNHh69DuVoiXcTHiNlQYsZTDWlq_TA>
> https://github.com/pat/thinking-sphinx/releases/tag/v3.3.0 
> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fpat%2Fthinking-sphinx%2Freleases%2Ftag%2Fv3.3.0=D=1=AFQjCNGTQohrWlNsNkBgQ_IFFcGiTo8UzQ>
> 
> If the error (or something like it) is still happening on v3.3.0, do let me 
> know!
> 
> Cheers,
> 
> — 
> Pat
> 
>> On 19 Jul 2017, at 8:10 am, Melody B <mel...@sassafras.coop <>> wrote:
>> 
>> I'm getting an error on configure:
>> 
>> Generating configuration to ~/myapp/config/development.sphinx.conf
>> rake aborted!
>> NoMethodError: undefined method `<<' for nil:NilClass
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:96:in
>>  `block in append_presenter_to_attribute_array'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:93:in
>>  `each'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:93:in
>>  `append_presenter_to_attribute_array'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:131:in
>>  `prepare_for_render'
>> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:65:in
>>  `render'
>> 
>> I have a few weird things going on in my setup that are (probably?) having 
>> an impact.
>> My database is postgres has uuid primary keys
>> I defined a custom field that generates a bigint in a before_create callback 
>> called secondary_id -- this seemed like the best way to provide sphinx with 
>> something it could index on but if this is the wrong way to go about keeping 
>> my uuid primary keys i'm open to alternative suggestions
>> I set secondary_id as the primary key for sphinx in the index definition 
>> with the follow

Re: [ts] Configuration fails with error "undefined method '<<' for nil:NilClass"

[Pat Allan]

Hi Melody,

There was a couple of commits related to fixing alternative primary keys that 
are part of v3.3.0 - I realise if you’re not familiar with this gem or Sphinx 
it could be daunting, but I’d recommend upgrading if at all possible.

The release notes for v3.2.0 and v3.3.0 (the only releases since v3.1.4) cover 
off notable changes - I’d say there’s a good chance you won’t hit any issues 
though:
https://github.com/pat/thinking-sphinx/releases/tag/v3.2.0 

https://github.com/pat/thinking-sphinx/releases/tag/v3.3.0 


If the error (or something like it) is still happening on v3.3.0, do let me 
know!

Cheers,

— 
Pat

> On 19 Jul 2017, at 8:10 am, Melody B  > wrote:
> 
> I'm getting an error on configure:
> 
> Generating configuration to ~/myapp/config/development.sphinx.conf
> rake aborted!
> NoMethodError: undefined method `<<' for nil:NilClass
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:96:in
>  `block in append_presenter_to_attribute_array'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:93:in
>  `each'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:93:in
>  `append_presenter_to_attribute_array'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:131:in
>  `prepare_for_render'
> ~/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/thinking-sphinx-3.1.4/lib/thinking_sphinx/active_record/sql_source.rb:65:in
>  `render'
> 
> I have a few weird things going on in my setup that are (probably?) having an 
> impact.
> My database is postgres has uuid primary keys
> I defined a custom field that generates a bigint in a before_create callback 
> called secondary_id -- this seemed like the best way to provide sphinx with 
> something it could index on but if this is the wrong way to go about keeping 
> my uuid primary keys i'm open to alternative suggestions
> I set secondary_id as the primary key for sphinx in the index definition with 
> the following code:
> ThinkingSphinx::Index.define :answer, with: :active_record, primary_key: 
> :proxy_id do ... end
> I'm honestly not sure what's going on here -- I tried poking around in the 
> code to sort out why this was happening and I tried guarding for nil on that 
> line allows configuring to finish, but it predictably continues to explode 
> during actual operation when I create a new instance of :answer -- I knew 
> that was a longshot, but any help would be appreciated. I'm not particularly 
> experienced with sphinx or thinking-sphinx, I inherited this project after it 
> was already in place (the move to uuid primary keys is fairly recent, and 
> this is the last major blocker) so just let me know if you need any 
> additional info or context.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Searching on special characters

[Pat Allan]

Hi Mark,

I’ve just tested this in one of my own apps, and I can get it working if I add 
( and ) to my charset_table - the full value in config/thinking_sphinx.yml is 
the following:

charset_table: 0..9, A..Z->a..z, _, a..z, U+410..U+42F->U+430..U+44F, 
U+430..U+44F, U+401->U+451, U+451, (, )

(This is Sphinx’s default, plus the two parentheses). Doing this, then running 
a rebuild, meant I got the expected search results. Is this what you’d tried? 
If it didn’t work, I’m keen to know which version of Sphinx you’re using.

Cheers,

— 
Pat

> On 25 Jun 2017, at 7:16 am, Mark Thomson  > wrote:
> 
> I'm trying to figure out how to search on a term that has parentheses in it, 
> e.g. like "abc (def)".
> 
> From Tom Copeland 
> ,
>  I gather these are sphinx special characters and so I've escaped them as he 
> advises. This prevents the search from crashing and is a reasonably workable 
> solution for me, but it's not actually searching on the parenthesis 
> characters, e.g. searching on "abc def" returns an indexed value "abc (def)" 
> as well. So it seems like the parentheses are just being ignored.
> 
> I've also tried adding a char_table that includes the parenthesis characters 
> to thinking_sphinx.yml, but this doesn't seem to have any effect.
> 
> Any clues?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Is random still possible? Have been unable to find a way to make it happen

[Pat Allan]

Hi Gordon

Sorting randomly is certainly an option, using :order => "RAND()”.

However, it seems that it only works as the lone ordering argument. In my tests 
locally, Sphinx throws an error when trying to order by attributes and RAND(), 
so I have a feeling you’re not going to be able to do what you want in a single 
query. Perhaps worth asking the Sphinx team via their forum, though I wouldn’t 
expect any changes in the near future to Sphinx’s behaviour:
http://sphinxsearch.com/forum/ 

Cheers,

— 
Pat

> On 29 Jun 2017, at 5:19 am, Gordon B. Isnor  > wrote:
> 
> We're using TS ~ 3.3.0, with SQL-backed indices
> 
> A client is looking for results with the following sort order:
> 
> - records with images first in random order
> - then records with no images, also in random order
> 
> I see that older versions of TS and Sphinx offered a @random option, and it 
> looks like Sphinx still has an extended sort mode, but in reviewing the 
> documentation and code case I can't find anything pertaining to sort mode. 
> I've also tried selecting RANDOM() as r and then sorting on that, but also 
> could not get that working. 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Wordbreaker for thinking sphinx

[Pat Allan]

Hi Stephan,

At this point in time there’s no wordbreaker integration with Thinking Sphinx. 
I’m certainly open to that happening, and patches are definitely welcome. 
Otherwise, you should be able to manually call indexer and wordbreaker yourself 
from within your Ruby code.

For anyone who’s pondering coding up a patch, here’s what I envisage is needed:

* Adapt indexer calls in Riddle to allow for buildstops and buildfreqs 
arguments.
* Add ability to call wordbreaker from within Riddle (like it already does for 
indexer and searchd).
* Add corresponding integration into Thinking Sphinx using the new Riddle 
functionality.

I think this would all be reasonably simple, though certainly not a five minute 
job.

— 
Pat

> On 28 Mar 2017, at 12:36 am, stephan.forte...@relendo.com 
>  wrote:
> 
> Hi there,
> 
> Is it possible to use the sphinx wordbreaker tool  
> with
>  thinking sphinx and how? I couldn't find anything in the thinking sphinx 
> documentation.
> 
> Thanks,
> Stephan
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Help updating from TS 2.0/2.1 to 3.0

[Pat Allan]

Hi Sven,

>From what I can remember of TS v2, the MVA string attributes were actually 
>transformed under the hood to CRC32 values, which was quite hawkish, hence TS 
>v3 onwards taking the simpler approach, as you’ve found.

Especially given you’re going through a has_many association, I would recommend 
using the primary key from that association as the facet value (thus, 
integers), and then translate ids to strings in your interface as required. I 
realise that means more work for you, but I think skipping around the 
transformation and related challenges is a better way to go overall.

Good luck with the rest of the updating!

— 
Pat

> On 21 Mar 2017, at 7:30 am, 'Sven Riedel' via Thinking Sphinx 
> > 
> wrote:
> 
> Hi,
> yes you read correctly, this is one for the archeologists :). 
> 
> I've inherited a project at work where the current state of the code is a tad 
> on the dated side, and unfortunately none of the original team members are 
> around to bug with questions. Since I'm a Sphinx novice, I'm at a bit of a 
> loss when it comes to non-trivial issues.
> 
> So far I've extracted the index definition of the model in question to 
> app/indices according to 
> http://freelancing-gods.com/thinking-sphinx/indexing.html#attributes 
>  . 
> However, there are some attribute definitions that work fine with TS 2.0 (and 
> 2.1) but TS 3 complains about them being string typed MVAs. I haven't been 
> able to figure out if TS2 supported string MVAs and that feature got cut in 
> TS3, or if some transformation of the attribute definition took place in the 
> background that doesn't happen anymore.
> 
> The offending attribute definitions have the following format in TS2:
> 
> has (:), facet: true, type: :string, as: 
> 
> 
> I've transformed them to the TS3 syntax accordingly:
> ThinkingSphinx::Index.define ...
>   has  ., facet: true, type: :string, as: 
> 
> end
> 
> Excluding the string type, the definition itself looks ok to me.
> 
> Are there any workaround for these situations beyond transforming all the 
> attribute values with CRC32() and handing the queries appropriately? Since I 
> don't know the application logic in detail and more importantly the actual 
> reasoning behind it, I'm hesitant to make big changes in the code base that 
> could have behavioural changes as a side effect.
> 
> We're running Sphinx 2.1.9 on the server side, if that makes any difference.
> 
> Any help is appreciated.
> 
> Thanks,
> Sven
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] records removed from core and not added in delta indexes..... +++ .....*.temp files in indices_location

[Pat Allan]

Hi Kyle,

You’re spot on with the temp files being the cause of the problem.

Firstly, to explain the temp (guard) files: they’ve been part of Thinking 
Sphinx since 3.1.0 and are a simple locking mechanism to ensure Thinking Sphinx 
doesn’t try to process the same index concurrently. Since 3.1.2, the files 
should be cleaned up on exceptions, but yes, as you’ve said more serious 
crashes might skip that logic? I can’t think of other situations, to be honest…

When indexing is requested and the index is already locked, there will be a 
‘warn’ level log to note this in the Rails logs:
Guard file for index advisor_delta exists, not indexing

… so, perhaps hunting through those logs and anything Sidekiq outputs might 
provide clues? If there are segfaults or similar, surely they’d be in the logs?

One related thing that may help is adding a uniqueness constraint to Sidekiq, 
at least for the ts_delta queue, to avoid concurrent indexing of any given 
delta.

If I think of anything else that may help, I’ll let you know.

— 
Pat

> On 17 Feb 2017, at 11:19 am, Kyle Ginavan  > wrote:
> 
> We are experiencing some issues with deltas on our production server. Once a 
> record has been updated it can no longer be queried. We are unable to query 
> it using the updated record by the old or the new name. The record appears to 
> be gone from sphinx. When the issue starts happening it doesn't happen to all 
> record types, just a few of them... mostly the larger ones. Furthermore, once 
> the issue starts happening a full index and rotate does not fix it. We have 
> to delete these *.temp files for the indices that stopped working. Once the 
> files are removed everything starts working again. Unfortunately we do not 
> have steps to recreate it, however, one hypothesis could be due to a large 
> load. We are also using sidekiq to queue up the deltas and another hypothesis 
> is that unrelated code in other sidekiq jobs has a segfault error or other 
> really bad crash other than an exception. Obviously those crashes need 
> resolved too. 
> 
> When this error starts the sphinx.log file will stop outputting the below and 
> once we remove the *.temp files they show up again.
> [Thu Feb 16 18:10:23.185 2017] [17066] rotating index 'advisor_delta': started
> [Thu Feb 16 18:10:23.186 2017] [17066] rotating index 'advisor_delta': success
> 
> Therefore, all the evidence appears that items are being removed from the 
> core index and not added to the delta index. One idea is that it happens once 
> these *.temp files show up in the indices_location. What are these *.temp 
> files? Is there another log file or log file setting that would help us 
> identify the issue here? Any other data points we should be looking at?
> 
> ==TECH & SETTINGS==
> 
> sphinx: 2.1.5
> thinking sphinx: 3.1.3
> rails: 4.2.7.1
> ruby: 2.2.3
> 
> mem_limit: 1024M
> 64bit_timestamps: true
> enable_star: true
> min_infix_len: 2
> indices_location: /data/sphinx/indexes
> configuration_file: /data/agencieshq/shared/config/sphinx.conf
> log: /data/agencieshq/shared/log/sphinx.log
> query_log: /data/agencieshq/shared/log/sphinx.query.log
> pid_file: /data/agencieshq/shared/pids/sphinx.pid 
> binlog_path: /data/sphinx/binlog
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] New additions not appearing in search results

[Pat Allan]

Great to hear you’ve made some progress on this issue - hopefully the database 
challenges are easy enough to solve!

— 
Pat

> On 2 Dec. 2016, at 12:00 am, Simon <simon.wilkin...@gmail.com 
> <mailto:simon.wilkin...@gmail.com>> wrote:
> 
> The value for Status.active.id is the same as it has been. I also double 
> checked this and older entries included in the index have the same status_id 
> as the newer entries not being included. 
> 
> The configuration file is being updated when I run a full ts:index or 
> ts:rebuild.
> 
> The sql_query value is currently this:
> sql_query = SELECT SQL_NO_CACHE `entries`.`id` * 1 + 0 AS `id` , 
> `entries`.`title` AS `title`, `entries`.`body` AS `body`, `entries`.`id` AS 
> `sphinx_internal_id`, 3940594292 AS `class_crc`, 0 AS `sphinx_deleted`, 
> `entries`.`journal_id` AS `journal_id`, 
> UNIX_TIMESTAMP(`entries`.`created_at`) AS `created_at`, 
> UNIX_TIMESTAMP(`entries`.`opened_at`) AS `opened_at`, `entries`.`status_id` 
> AS `status_id` FROM `entries`WHERE `entries`.`id` >= $start AND 
> `entries`.`id` <= $end AND status_id = 1 GROUP BY `entries`.`id`  ORDER BY 
> NULL
> 
> If I change that to simply do a 'SELECT COUNT(*) FROM `entries` WHERE 
> status_id = 1 GROUP BY `entries`.`id`  ORDER BY NULL' the count I get back is 
> 19,940,635.   Which actually matches the number of records that I'm seeing on 
> the ts:reindex call.  
> 
> The DB I run the indexing and searching on is actually a slave DB, and when I 
> run the same query on the master I get back a much larger number 
> (20,527,906). So it seems the issue may not be with Sphinx at all, but with 
> my master-slave setup (which is even more confusing because the slave status 
> shows as 0 seconds behind the master). Anyways, looks like I need to turn my 
> attention to non-sphinx issues.
> 
> Thanks again Pat for taking the time to respond, and for getting me thinking 
> about some different possibilities. It's greatly appreciated!
> 
> Cheers,
> Simon
> 
> 
> On Wednesday, 30 November 2016 07:47:18 UTC-5, Pat Allan wrote:
> Hmm, this is certainly an odd one! Is it possible the Status.active.id 
> <http://status.active.id/> value has changed?
> 
> The ts:reindex task *only* reindexes the data. ts:index, however, will both 
> regenerate the configuration file and reindex the data. Given you’ve been 
> running the former, that would explain why the delta indices are still 
> present in the generated configuration file. That said, running ts:rebuild 
> should regenerate the configuration file as well, so I’m wondering if that 
> file isn’t being updated for some reason? So, that’s where my next focus for 
> debugging on the server would be…
> 
> … and if it *is* regenerating correctly, and the deltas are now removed, the 
> next question is: is the generated sql_query value for the source correct? 
> Can you take that query and modify it to use COUNT(*) and confirm how many 
> records it matches against?
> 
> Also, just for reference: which version of the Thinking Sphinx gem are you 
> using?
> 
>> On 30 Nov. 2016, at 11:41 pm, Simon <simon.w...@gmail.com > 
>> wrote:
>> 
>> Hi Pat, 
>> 
>> Thanks so much for the response! The number of records actually does not 
>> appear to fully match, as a count on active entries returns 20,515,798.  
>> Also, after having my normal cron job that re-indexes run this morning, I 
>> noticed that the number of records collected is the exact same as before 
>> (19,940,635).  Is there a limit to the number of records Sphinx can handle, 
>> or any other common scenarios that could be preventing new entries from 
>> getting included in the indexing?
>> 
>> As for the delta indexes, we actually removed these a while ago from the 
>> index definition as they were causing some headaches.  Our configuration 
>> file still includes the delta block, but this has never seemed to be an 
>> issue in indexing.  I could remove the delta info from the config file 
>> (something I've actually been meaning to do), but I didn't want to introduce 
>> more variables into what might have changed while trying to trouble shoot 
>> this issue. 
>> 
>> Here is the search call, even though the record counts don't match, just in 
>> case it is helpful at all in continuing to try and figure this out:
>> 
>> filters = {
>>   :journal_id => journal_ids,
>>   :status_id => Status.active.id <http://status.active.id/>
>>   }
>> # Check to see if we are ordering in a specific way
>> params[:order] ||= '@relevance DESC'
>> case params[:order]
>> when 'c

Re: [ts] New additions not appearing in search results

[Pat Allan]

Hmm, this is certainly an odd one! Is it possible the Status.active.id value 
has changed?

The ts:reindex task *only* reindexes the data. ts:index, however, will both 
regenerate the configuration file and reindex the data. Given you’ve been 
running the former, that would explain why the delta indices are still present 
in the generated configuration file. That said, running ts:rebuild should 
regenerate the configuration file as well, so I’m wondering if that file isn’t 
being updated for some reason? So, that’s where my next focus for debugging on 
the server would be…

… and if it *is* regenerating correctly, and the deltas are now removed, the 
next question is: is the generated sql_query value for the source correct? Can 
you take that query and modify it to use COUNT(*) and confirm how many records 
it matches against?

Also, just for reference: which version of the Thinking Sphinx gem are you 
using?

> On 30 Nov. 2016, at 11:41 pm, Simon <simon.wilkin...@gmail.com 
> <mailto:simon.wilkin...@gmail.com>> wrote:
> 
> Hi Pat, 
> 
> Thanks so much for the response! The number of records actually does not 
> appear to fully match, as a count on active entries returns 20,515,798.  
> Also, after having my normal cron job that re-indexes run this morning, I 
> noticed that the number of records collected is the exact same as before 
> (19,940,635).  Is there a limit to the number of records Sphinx can handle, 
> or any other common scenarios that could be preventing new entries from 
> getting included in the indexing?
> 
> As for the delta indexes, we actually removed these a while ago from the 
> index definition as they were causing some headaches.  Our configuration file 
> still includes the delta block, but this has never seemed to be an issue in 
> indexing.  I could remove the delta info from the config file (something I've 
> actually been meaning to do), but I didn't want to introduce more variables 
> into what might have changed while trying to trouble shoot this issue. 
> 
> Here is the search call, even though the record counts don't match, just in 
> case it is helpful at all in continuing to try and figure this out:
> 
> filters = {
>   :journal_id => journal_ids,
>   :status_id => Status.active.id
>   }
> # Check to see if we are ordering in a specific way
> params[:order] ||= '@relevance DESC'
> case params[:order]
> when 'cad'
>   order = 'created_at DESC'
> when 'ca'
>   order = 'created_at ASC'
> else
>   order = '@relevance DESC'
> end
> entries = Entry.search params[:criteria], :with => filters, :sort_mode => 
> :extended, :order => order
> 
> Thanks again,
> Simon
> 
> On Wednesday, 30 November 2016 07:29:36 UTC-5, Pat Allan wrote:
> Hi Simon,
> 
> I guess the first place I’d start is by verifying the number of records 
> you’re expecting Sphinx to index. The log you shared says 19,940,635 - does 
> that match Entry.count(:conditions => {:status_id => Status.active.id 
> <http://status.active.id/>})?
> 
> Also: the indexing output suggests there’s a delta index, but that’s not in 
> the index definition - removed for brevity?
> 
> And if the counts match, then can you share the search call you’re running to 
> confirm newer records are not appearing?
> 
> Cheers,
> 
> — 
> Pat
> 
>> On 30 Nov. 2016, at 12:40 am, Simon <simon.w...@gmail.com > 
>> wrote:
>> 
>> Hi,
>> 
>> I'm having an issue that just started recently. Indexing appears to complete 
>> successfully, but new entries are not appearing in search results (older 
>> entries appear). 
>> 
>> This seems to have started after I tried a rake ts:rebuild instead of what I 
>> normally used (rake ts.reindex). I have since switched back to a reindex, 
>> but still nothing new seems to be getting picked up.  Unfortunately I am 
>> running an older version of Ruby (1.8.7), Rails (2.3.18) and Sphinx (Sphinx 
>> 1.10-beta (r2420)). 
>> 
>> My model definition is as follows:
>> 
>> define_index do
>> indexes title
>> indexes body
>> has journal_id
>> has created_at, opened_at, status_id
>> 
>> where "status_id = #{Status.active.id <http://status.active.id/>}"
>>   end
>> 
>> The output of calling rake ts:reindex is:
>> 
>> Sphinx 1.10-beta (r2420)
>> 
>> Copyright (c) 2001-2010, Andrew Aksyonoff
>> 
>> Copyright (c) 2008-2010, Sphinx Technologies Inc (http://sphinxsearch.com 
>> <http://sphinxsearch.com/>)
>> 
>> 
>> 
>> using config file 
>> '/home/ubuntu/rails/penzu/confi

Re: [ts] New additions not appearing in search results

[Pat Allan]

Hi Simon,

I guess the first place I’d start is by verifying the number of records you’re 
expecting Sphinx to index. The log you shared says 19,940,635 - does that match 
Entry.count(:conditions => {:status_id => Status.active.id})?

Also: the indexing output suggests there’s a delta index, but that’s not in the 
index definition - removed for brevity?

And if the counts match, then can you share the search call you’re running to 
confirm newer records are not appearing?

Cheers,

— 
Pat

> On 30 Nov. 2016, at 12:40 am, Simon  > wrote:
> 
> Hi,
> 
> I'm having an issue that just started recently. Indexing appears to complete 
> successfully, but new entries are not appearing in search results (older 
> entries appear). 
> 
> This seems to have started after I tried a rake ts:rebuild instead of what I 
> normally used (rake ts.reindex). I have since switched back to a reindex, but 
> still nothing new seems to be getting picked up.  Unfortunately I am running 
> an older version of Ruby (1.8.7), Rails (2.3.18) and Sphinx (Sphinx 1.10-beta 
> (r2420)). 
> 
> My model definition is as follows:
> 
> define_index do
> indexes title
> indexes body
> has journal_id
> has created_at, opened_at, status_id
> 
> where "status_id = #{Status.active.id}"
>   end
> 
> The output of calling rake ts:reindex is:
> 
> Sphinx 1.10-beta (r2420)
> 
> Copyright (c) 2001-2010, Andrew Aksyonoff
> 
> Copyright (c) 2008-2010, Sphinx Technologies Inc (http://sphinxsearch.com 
> )
> 
> 
> 
> using config file 
> '/home/ubuntu/rails/penzu/config/pandora_readonly.sphinx.conf'...
> 
> indexing index 'entry_core'...
> 
> collected 19940635 docs, 34535.3 MB
> 
> WARNING: sort_hits: merge_block_size=224 kb too low, increasing mem_limit may 
> improve performance
> 
> sorted 6182.9 Mhits, 100.0% done
> 
> total 19940635 docs, 34535317325 bytes
> 
> total 16953.201 sec, 2037097 bytes/sec, 1176.21 docs/sec
> 
> indexing index 'entry_delta'...
> 
> collected 0 docs, 0.0 MB
> 
> total 0 docs, 0 bytes
> 
> total 0.155 sec, 0 bytes/sec, 0.00 docs/sec
> 
> skipping non-plain index 'entry'...
> 
> total 92992 reads, 1534.381 sec, 228.9 kb/call avg, 16.5 msec/call avg
> 
> total 40583 writes, 192.954 sec, 1042.6 kb/call avg, 4.7 msec/call avg
> 
> rotating indices: succesfully sent SIGHUP to searchd (pid=3802).
> 
> 
> So it all appears successful, but no new results appear. So for instance, if 
> i add a new entry, and then call reindex, that entry is not found in search 
> results. But an entry with the same search term from a month ago does appear 
> in the results.
> 
> I have tried a complete rake ts:index, I have tried deleting all of the 
> generated index files (entry_core.spp, entry_core.spi, etc.) but nothing 
> seems to make a difference. Does anybody have any ideas what might be 
> happening here, or any other suggestions for what I can try?
> 
> Thanks,
> Simon
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Real time index with association not showing query results properly

[Pat Allan]

Thanks for those details Terence, and great to know the searches are working 
when you specify the indices (though I’m surprised it’s different when 
specifying models instead - are you using STI for any of these indexed models?)

Regarding the filters, there’s a few things to note:

The :with option is for attributes, and yet you’re only defining fields in your 
index definitions (the indexes method is for fields, the has method is for 
attributes). Some logic that may help for deciding which to use:

Is the data something you want to match against search queries (e.g. 
params[:search] in your earlier example)? Then it should be a field.
Do you also want to be able to sort by this data? Then add :sortable => true.
But, if you only want to sort by that data, or you’re filtering or grouping by 
it - which is often the case for non-string data like floats, integers and 
timestamps - then they should be attributes (which are sortable by their very 
nature - thus, the :sortable option is not needed).

Alongside that knowledge, it’s also worth noting that NULL values are stored as 
0 by Sphinx.

Now, in the case of your country_id filter, there’s a few different approaches. 
For all of them, though, I’m presuming country_id is an integer, and thus is an 
attribute in your index definitions.

If you’re filtering by country, do you want all Source values to be returned?

Add a country_id method to your Source model, which returns 0, and a 
corresponding attribute.
Filter on both the supplied country_id value (as an integer), and 0: :with => 
{:country_id => [params[:country].to_i, 0]}

Or, if you want no Source values to be returned, you still add that method and 
attribute, but filter only by the supplied country value.

However, if some providers don’t have countries, their country_id values will 
be NULL/0, so the first approach won’t work. Instead, I’d recommend using 
floats and then have Source country_ids to be -1.0, and update your filter 
accordingly. I’m pretty sure integers in Sphinx are unsigned, hence the need to 
switch to floats.

Hope this helps!

— 
Pat

> On 9 Sep 2016, at 12:47 AM, Terence Leong  wrote:
> 
> Ah noted on the lower-case symbol will do adjustments accordingly!
> 
> Here are the other index definition:
> ThinkingSphinx::Index.define :PocketWifi, :with => :real_time do
>   # fields
>   indexes number_of_user, :sortable => true, :match_mode => :any, :sort_mode 
> => :desc
>   indexes origin_country, :sortable => true, :match_mode => :any, :sort_mode 
> => :desc
>   indexes destination_country, :sortable => true, :match_mode => :any, 
> :sort_mode => :desc
> 
> ThinkingSphinx::Index.define :SimSize, :with => :real_time do
>   # fields
>   indexes sim_name, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes sim_type, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
> 
> ThinkingSphinx::Index.define :Rechargeable, :with => :real_time do
>   # fields
>   indexes recharge_quota, :sortable => true, :match_mode => :any, :sort_mode 
> => :desc
>   indexes recharge_price, :sortable => true, :match_mode => :any, :sort_mode 
> => :desc
>   indexes recharge_desc, :sortable => true, :match_mode => :any, :sort_mode 
> => :desc
> 
> I figured it was a problem with the where clause during the query and made 
> some adjustments.
> It is working but can I assign thinking sphinx search to only search through 
> an index?
> 
> @results = ThinkingSphinx.search(
> :with => { :country_id => "#{params[:country]}", :plan_day => 
> "#{params[:duration]}"}, 
> :select => '*',
> :indices => ['Provider_core', 'Source_core'] 
>   )
> 
> provider_core contains country but source_core does not and it's giving me an 
> error due to that, any way I can simplify this?
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Multiple Indexes in real time index

[Pat Allan]

Hi Pulkit

You’re right, the where method only works in SQL-backed indices. There’s two 
steps to work around this for real-time indices (though please note I’ve not 
tested this!):

* Use scope in the index definitions to define the ActiveRecord scope used by 
ts:generate/ts:regenerate:

  scope { MyModel.where("id % 2 = 0")

* Have a custom callback to direct inserts/updates to the correct index 
(instead of the standard after_save approach covered in the documentation):

  after_save :update_sphinx

  def update_sphinx
indices = 
ThinkingSphinx::Configuration.instance.indices_for_references(:my_model)
if id % 2 = 0
  index = indices.detect { |index| index.name == ‘my_model_index0' }
else
  index = indices.detect { |index| index.name == ‘my_model_index1' }
end

ThinkingSphinx::RealTime::Transcriber.new(index).copy self
  end

Give this a shot and let us know how it goes!

— 
Pat

> On 10 Sep 2016, at 12:51 AM, Pulkit Sharma  wrote:
> 
> Hi,
> 
> 
> 
> The machine I am running searchd on is a dual-core one, so I plan to create a 
> distributed index for real-time indexes and set the appropriate value for 
> ‘dist_threads’.
> 
> 
> 
> is this the correct way to create index so that I can utilise both the cores ?
> 
> 
> 
> ThinkingSphinx::Index.define :my_model, :name => ‘my_model_index0', :with => 
> :real_time do
> 
> where 'id % 2 = 0'
> 
> indexes name
> 
> end
> 
> 
> 
> ThinkingSphinx::Index.define :my_model, :name => ‘my_model_index1', :with => 
> :real_time do
> 
> where 'id % 2 = 1'
> 
> indexes name
> 
> end
> 
> 
> 
> The above generates two identical indexes(it seems the where condition is not 
> being used, because this is not backed by ActiveRecord) , which is not what I 
> want. I want to split the data in half among them. How to split the index 
> conditionally ?
> 
> 
> 
> Thank You
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Real time index with association not showing query results properly

[Pat Allan]

Hmm, that’s interesting, though not sure if it’s part of the problem. We shall 
see, though!

You shared two index definitions, but you’re searching on several models. Do 
you get the expected results on any of the models reliably? Can you share all 
index definitions here?

Also, it’s worth noting that the reference to the model in an index definition 
should be a lower-case symbol - e.g. :provider, :source. This also applies when 
adding the callback to those models:

  after_save ThinkingSphinx::RealTime.callback_for(:provider)

Not sure if that’s related either, mind you… but this is an odd problem, so I’m 
really not sure at all why it’s happening!

— 
Pat

> On 8 Sep 2016, at 12:37 PM, Terence Leong  wrote:
> 
> Ah sorry for the confusion, by entries I mean the data in the database that 
> has the parent id assigned like:
> 
> Source 1: contains the id of a provider
> Source 2: does not contain the id of a provider
> 
> the problem only appears on those who contains the ID of a provider.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Real time index with association not showing query results properly

[Pat Allan]

Hi Terence,

Thanks for those extra details. Just to be clear, though: can you change your 
Gemfile to use the latest develop commit - there may be some code changes since 
the 3.2.0 release that could help matters.

Also: I don’t understand what you mean with ‘entries that have the id of the 
parent’?

— 
Pat

> On 8 Sep 2016, at 3:29 AM, Terence Leong  wrote:
> 
> Hello Pat, 
> I'm using the latest version of sphinx (2.2.11) and thinking sphinx (gem 
> 'thinking-sphinx', '~> 3.2.0')
> I have ran rake ts:regenerate multiple times to no avail and I'm solely using 
> real time indices only.
> 
> I have deleted the directories as you suggested and ran rake ts:regenerate 
> and it's still returning me the same results however I noticed that it only 
> affects the entries that have the id of the parent. Thanks for the heads up 
> on the match_mode and sort_mode!
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Real time index with association not showing query results properly

[Pat Allan]

Hi Terence

A few things to try:

* Can you update to the latest Thinking Sphinx code (from the develop branch in 
the GitHub repo):

  gem ‘thinking-sphinx’, ‘~> 3.2.0’,
:git=> ‘git://github.com/pat/thinking-sphinx.git' 
,
:branch => ‘develop’
:ref=> ‘cf16d18285'

* Once you’ve updated, can you run ts:regenerate and see if that helps?
* Are you mixing SQL-backed indices and real-time indices? I’d recommend only 
using one or the other.
* If none of this helps, can you try running `rake ts:stop`, then delete the 
db/sphinx and tmp/binlog directories, then `rake ts:regenerate` again?
* Which version of Sphinx are you using?

Unrelated to your problem, but the match_mode and sort_mode arguments are not 
used in index definitions. The sort_mode argument only applies in search calls, 
and match_mode was only for search calls in TS v1/v2. v3 uses the SphinxQL 
protocol (as you can see in the logs - it’s very similar to SQL), and that 
always uses the extended match mode.

Cheers

— 
Pat

> On 7 Sep 2016, at 1:40 PM, Terence Leong  wrote:
> 
> Hello, recently I have just started using sphinx and thinking sphinx on rails 
> 5.0. I have created a project to test it out.
> SQL backed index worked fine however I faced many problems when I switched to 
> real time index.
> The first screen shot shows that the wildcard query returns me all the 
> objects in the index while the second screen shot returns me duplicated 
> results of the same object.
> The third screen shot shows that it returned the results I wanted. These are 
> my code please advise.
> 
> index controller:
> @results = ThinkingSphinx.search(
> "#{params[:search]}*",
> :select => '*, weight() as w', :classes => [Provider, Source, 
> Rechargeable, SimSize, PocketWifi], :order => 'w DESC'
> )
> 
> ThinkingSphinx::Index.define :provider, :with => :real_time do
>   # fields
>   indexes provider_name, :sortable => true, :match_mode => :any, :sort_mode 
> => :desc
>   indexes provider_country, :sortable => true, :match_mode => :any, 
> :sort_mode => :desc
>   indexes source_info, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
> 
>   set_property :enable_star => 1 #enables wildcard
>   set_property :min_infix_len => 2 #define wildcard query length
> 
> ThinkingSphinx::Index.define :Source, :with => :real_time do
>   # fields
>   indexes plan_name, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes max_speed, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes price, :sortable => true, :match_mode => :any, :sort_mode => :desc
>   indexes remark, :sortable => true, :match_mode => :any, :sort_mode => :desc
>   indexes tethering, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes where_to_buy, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes sim_disposal, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes fair_usage, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes activation, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes comm_standard, :sortable => true, :match_mode => :any, :sort_mode 
> => :desc
>   indexes plan_quota, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes plan_day, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes simsize_info, :sortable => true, :match_mode => :any, :sort_mode => 
> :desc
>   indexes rechargeable_info, :sortable => true, :match_mode => :any, 
> :sort_mode => :desc
>   indexes pocketwifi_info, :sortable => true, :match_mode => :any, :sort_mode 
> => :desc
> 
>   set_property :enable_star => 1 #enables wildcard
>   set_property :min_infix_len => 2 #define wildcard query length
> 
> class Provider < ApplicationRecord
>   has_many :Source
>   after_save ThinkingSphinx::RealTime.callback_for(:Provider)
> 
>   def source_info
>   Source.pluck(:plan_name, :max_speed, :price, :remark, :tethering, 
> :where_to_buy, :sim_disposal, :fair_usage, :activation, :comm_standard, 
> :plan_quota, :plan_day).join(' ')
>   end
> end
> 
> class Source < ApplicationRecord
>   belongs_to :Provider
>   has_many :SimSize
>   has_many :Rechargeable
>   has_many :PocketWifi
>   after_save ThinkingSphinx::RealTime.callback_for(:Source)
> 
>   def simsize_info
>   SimSize.pluck(:sim_name, :sim_type).join(' ')
>   end
> 
>   def rechargeable_info
>   Rechargeable.pluck(:recharge_quota, :recharge_price, 
> :recharge_desc).join(' ')
>   end
> 
>   def pocketwifi_info
>   PocketWifi.pluck(:number_of_user, :origin_country, 
> :destination_country).join(' ')
>   end
> end
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> 

Re: [ts] Re: ThinkingSphinx Has many through

[Pat Allan]

Hi Scott

Sounds like the `ts:regenerate` task might not be stopping the Sphinx daemon. 
Can you try the following:

* run `rake ts:stop`
* check that the daemon is not running using `ps aux | grep searchd` - you 
don’t want to see any searchd processes.
* manually kill any searchd processes if they are still running
* run `rake ts:regenerate`

Hope this sorts it out.

— 
Pat

> On 2 Aug 2016, at 3:05 AM, Scott Macri  wrote:
> 
> Thanks for your help, but unfortunately this causes a different error when 
> running 'rake ts:regenerate':
> ThinkingSphinx::SphinxError: unknown column: 'bbc_compliance_names' - REPLACE 
> INTO bbc_facility_core (id, `sphinx_internal_class_name`, `name`, 
> `bbc_compliance_names`, `sphinx_internal_id`, `sphinx_internal_class`, 
> `sphinx_deleted`, `name_sort`) VALUES ('all values for fields are actually 
> listed here')
> 
> On Friday, July 22, 2016 at 11:52:44 PM UTC-4, Scott Macri wrote:
> Hello,
> 
> I'm new to sphinx and am attempting a has_many_through.  I'm using Rails 4.2 
> and Ruby 2.2.  I'm getting the following error when running config.
> ThinkingSphinx::SphinxError: unknown column: 'bbc_compliances' - REPLACE INTO 
> bbc_facility_core
> 
> Here is my setup:
> 
> ThinkingSphinx::Index.define :bbc_facility, :with => :real_time do
>indexes name, sortable: true
>indexes bbc_compliances.name , :as => 
> :bbc_compliances
> end
> 
> class BbcCompliance < ActiveRecord::Base
>   has_many :bbc_facility_compliances, dependent: :destroy
>   has_many :bbc_facilities, through: :bbc_facility_compliances
> end 
> 
> class BbcFacility < ActiveRecord::Base
>   after_save ThinkingSphinx::RealTime.callback_for(:bbc_facility)
>   has_many :bbc_facility_compliances, dependent: :destroy
>   has_many :bbc_compliances, through: :bbc_facility_compliances
> end
> 
> Any help would be greatly appreciated.  Also, where can I find some 
> documentation on complex topics such as this one?  Thanks.
>   
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Duplicate records returned

[Pat Allan]

Hi Pratik

I’m still not sure what the cause of this problem is. There have been a 
threadsafe-related change in Thinking Sphinx 3.2.0 which may help, but I’m not 
convinced that’s related to what you’re dealing with.

One thing I’d be interested in seeing is the raw Sphinx results… if you fire up 
a mysql console (`mysql —host 127.0.0.1 —port 9306`) and enter the following 
query (make sure the id is for an existing record that you’re seeing 
duplicated):

  SELECT * FROM text_message_core WHERE sphinx_internal_id = 1;

Can you share the results of that?

Thanks,

— 
Pat

> On 5 Jul 2016, at 10:54 PM, pratik ganvir <ganvirpra...@gmail.com> wrote:
> 
> All the records are duplicated not only the impacted once. Every record is 
> returned twice. Temporary I tried to group them by column id using group_by 
> option but still they are returned twice.
>  
> Below are the version of gems and the program.
> Gem vesrions:
> thinking-sphinx (3.1.4)
> Rails 4.2.5
> ruby 2.2.2p95
> 
> Program version:
> Sphinx 2.2.10-id64-release
> 
> - Pratik
> 
> On Tuesday, 5 July 2016 18:06:57 UTC+5:30, Pat Allan wrote:
> You’re right, update_all doesn’t fire callbacks, and given you’re not using 
> the status column in your index, that shouldn’t be a problem.
> 
> The duplicate records is an odd thing though. Are you finding _every_ result 
> is duplicated? Only ones impacted by the cron job? Or are other records 
> duplicated as well?
> 
> Also: which version of Thinking Sphinx are you using?
> 
> — 
> Pat
> 
>> On 5 Jul 2016, at 9:20 PM, pratik ganvir <ganvir...@gmail.com > 
>> wrote:
>> 
>> Hi Pat,
>> 
>> Thanks for the reply. Here is my index definition.
>> 
>> ThinkingSphinx::Index.define :text_message, :with => :real_time do
>>  has contact_group_id, type: :integer
>>   has user_id, type: :integer
>>   has id, :as => :sms_id, type: :integer
>>   has sms_campeign_id, :as => :sms_campeign_id, type: :integer
>>   has delivery_schedule, :type => :timestamp, :sortable => true
>>   has created_at, :type => :timestamp, :sortable => true
>> end
>> 
>> I am suing update_all method in my cron which surely doesn't fire callbacks.
>> Here is the cron method that does that.
>> 
>> def self.update_system_errors
>>  sms = Sms.where('status not in (?) and (created_at between ? 
>> and ?) and job_id in 
>> (?)',[10,11],DateTime.now.beginning_of_day,DateTime.now.end_of_day,["Daily 
>> Message Limit Reached"])
>> sms.update_all('status = 10')
>> sms = Sms.where('status not in (?) and (created_at between ? and ?) and 
>> job_id in 
>> (?)',[10,11],DateTime.now.beginning_of_day,DateTime.now.end_of_day,["Sender 
>> ID Does not Exist or Pending or Route Invalid!","No Sufficient Credits""])
>> sms.update_all('status = 11')
>> end
>> 
>> Thanks,
>> Pratik Ganvir
>> 
>> On Sunday, 3 July 2016 16:46:43 UTC+5:30, Pat Allan wrote:
>> Hi Pratik
>> 
>> Can you share your index definition? Also, how are you updating the 
>> TextMessage records via your cron process? Is it in a manner that will fire 
>> ActiveRecord callbacks? And is this the only model you have with a Sphinx 
>> index?
>> 
>> Cheers
>> 
>> — 
>> Pat
>> 
>>> On 2 Jul 2016, at 1:45 PM, pratik ganvir <ganvir...@gmail.com <>> wrote:
>>> 
>>> Hi,
>>> 
>>> So I have a model called TextMessage which has attributes id and user_id. I 
>>> am using real time indices. 
>>> There is one cron process that updates the status field of TextMessage 
>>> records. When it does that, following query returns 2 records for each 
>>> database record. 
>>> 
>>> TextMessage.search(with: {user_id: current_user.id 
>>> <http://current_user.id/>},order: 'created_at desc',page: params[:page], 
>>> per_page: 10 )
>>> 
>>> When I rebuild the index then everything goes fine. But considering the 
>>> fact that I am using realtime indices I do not want to rebuild indexes 
>>> everytime.
>>> 
>>> Please help me to get through this.
>>> 
>>> Thanks and regards,
>>> Pratik
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Thinking Sphinx" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to thinking-sphi...@googlegroups.com <>.
>>> To post to this group, send email to thin

Re: [ts] Delta indexing post update_all

[Pat Allan]

Hi Sundar

Sorry for being so slow on responding to this.

Certainly, the code you’ve got there will not remove documents from the core 
index. What this means is that any queries that match old data in the core 
index will return results that you may not want. What you could do instead is 
the following:

  ids = Article.where().pluck :id
  
  Article.where(:ids => ids).update_all :delta => true
  ThinkingSphinx::Deltas::IndexJob.new(‘article_delta’).perform
  
  indices = 
ThinkingSphinx::Configuration.instance.indices_for_references(:article)
  indices.reject { |index| index.delta? }.each do |index|
ids.each do |id|
  ThinkingSphinx::Deltas::DeleteJob.new(
index.name, index.document_id_for_key(id)
  ).perform
end
  end

Note that this code should work with TS v3.0.4. A slightly simpler approach 
(which involves far fewer Sphinx queries) is available since v3.1.2:

  ids = Article.where().pluck :id
  
  Article.where(:ids => ids).update_all :delta => true
  ThinkingSphinx::Deltas::IndexJob.new(‘article_delta’).perform
  
  indices = 
ThinkingSphinx::Configuration.instance.indices_for_references(:article)
  indices.reject { |index| index.delta? }.each do |index|
ThinkingSphinx::Deletion.perform index, ids
  end

If you wanted to push this behaviour into Delayed Job instead, then I’d 
recommend creating a custom job class to do so (as the internals of 
ts-delayed-delta for deleting from the core index are written with single 
instances in mind).

Hope this helps!

— 
Pat

> On 5 Jul 2016, at 11:16 PM, Sundar Rajamanickam 
>  wrote:
> 
> Hi
> 
> If I do the following, how will the documents be removed from core index ( 
> because we do not invoke ThinkingSphinx::Deltas::DeleteJob ) ? Is there an 
> impact with not removing the documents from core index ?
> 
> Article.where().update_all("delta = 1")
> ThinkingSphinx::Deltas::IndexJob.new("article_delta").perform
> 
> 
> Also, if I only want to en-queue jobs in DJ queue post update_all, what 
> should be done ?
> 
> 
> Gem versions:
> 
> gem 'thinking-sphinx', '3.0.4'
> gem 'ts-delayed-delta', '~> 2.0.2'
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] ThinkingSphinx Has many through

[Pat Allan]

Hi Scott

I think there are two issues here to deal with:

Firstly, method references for fields and attributes in real-time indices must 
be able to chain together in a standard Ruby way (in your example, you’re 
calling `name` on `bbc_compliances`, which is an array/association and doesn’t 
have a `name` method). The way around this would be to add the following method 
to your BbcFacility model:

  def bbc_compliance_names
# Returning a string, as that is what Sphinx expects for fields
bbc_compliances.collect(&:name).join(‘ ‘)
  end

And then in your index definition:

  indexes bbc_compliance_names

This is discussed in the indexing documentation under the Fields section:
http://freelancing-gods.com/thinking-sphinx/indexing.html#fields 


The second thing is actually the cause of the error you’re seeing: Sphinx is 
unaware of this field, which means you need to run `rake ts:regenerate` to 
clear out the old index files, restart Sphinx, and then re-populate with the 
new schema. This is required every time you add/change/remove fields, 
attributes or indices - it’s the rough equivalent of `rake db:migrate`.

This is covered in the rake tasks documentation in the first section (on 
ts:generate), but certainly could be more prominent!
http://freelancing-gods.com/thinking-sphinx/rake_tasks.html 


Let me know if you’ve any further questions!

— 
Pat

> On 23 Jul 2016, at 4:40 AM, Scott Macri  wrote:
> 
> Hello,
> 
> I'm new to sphinx and am attempting a has_many_through.  I'm using Rails 4.2 
> and Ruby 2.2.  I'm getting the following error when running config.
> ThinkingSphinx::SphinxError: unknown column: 'bbc_compliances' - REPLACE INTO 
> bbc_facility_core
> 
> Here is my setup:
> 
> ThinkingSphinx::Index.define :bbc_facility, :with => :real_time do
>indexes name, sortable: true
>indexes bbc_compliances.name, :as => :bbc_compliances
> end
> 
> class BbcCompliance < ActiveRecord::Base
>   has_many :bbc_facility_compliances, dependent: :destroy
>   has_many :bbc_facilities, through: :bbc_facility_compliances
> end 
> 
> class BbcFacility < ActiveRecord::Base
>   after_save ThinkingSphinx::RealTime.callback_for(:bbc_facility)
>   has_many :bbc_facility_compliances, dependent: :destroy
>   has_many :bbc_compliances, through: :bbc_facility_compliances
> end
> 
> Any help would be greatly appreciated.  Also, where can I find some 
> documentation on complex topics such as this one?  Thanks.
>   
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Hitting ThinkingSphinx::SphinxError: Too many connections

[Pat Allan]

Hi Scott

Great to know TS is appreciated :)

I don’t think I’ve come across this issue before… and a search of the Sphinx 
forum doesn’t turn up any likely settings either. The default is to accept as 
many connections as possible 
, and with TS’s 
connection pooling, I think it should create a connection per thread - and 
it’ll likely hold onto each of those connections while those threads are 
present. Each connection, in turn, creates a searchd child process - so perhaps 
you’re running out of resources on your machine to handle a large number of 
children processes?

A potential way around this is to turn off TS’s persistent connections: thus, 
each searchd child process will be created as needed, but then released again:

  ThinkingSphinx::Connection.persistent = false

Now, this means that every single search request is going to have the TCP 
overhead of setting up the connection to Sphinx. If this situation where you’re 
updating all of your items is done via a rake task, perhaps it’s worth only 
trying that setting within the task?

Also: TS should generally be threadsafe - though that probably doesn’t apply to 
the persistent setting, as that’s expected to be set once and not touched after 
that.

Hope this helps!

— 
Pat


> On 16 Jul 2016, at 6:07 PM, sc...@appdata.com wrote:
> 
> Hi, 
> 
> To start I wanted to thank Pat for Thinking Sphinx - wonderful bit of open 
> source.
> 
> I have a fairly large pool of items - 2387868 - that are each processed by 1 
> or more queries.  Given that each query takes about .1 seconds to run I'm 
> using a threaded approach to run them (thread pool style architecture with 
> each item being popped off a redis queue).
> 
> What I'm seeing is a large number of ThinkingSphinx::SphinxError: Too many 
> connections.  Here's a full stack trace:
> 
> SELECT * FROM `unique_link_core` WHERE MATCH('@site_id 37 @url_no_www 
> *\"debit ATM\"*') AND `sphinx_deleted` = 0 LIMIT 0, 20
> ThinkingSphinx::SphinxError: Too many connections - SELECT * FROM 
> `unique_link_core` WHERE MATCH('@site_id 37 @url_no_www *\"debit ATM\"*') AND 
> `sphinx_deleted` = 0 LIMIT 0, 20; SHOW META
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:93:in
>  `rescue in query'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:96:in
>  `query'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:77:in
>  `query_all'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search/batch_inquirer.rb:17:in
>  `block in results'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:39:in
>  `block in take'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/innertube-1.1.0/lib/innertube.rb:138:in
>  `take'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:37:in
>  `take'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search/batch_inquirer.rb:16:in
>  `results'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/middlewares/inquirer.rb:9:in
>  `block in call'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in
>  `block in instrument'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/activesupport-4.1.4/lib/active_support/notifications/instrumenter.rb:20:in
>  `instrument'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in
>  `instrument'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/logger.rb:3:in
>  `log'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/middlewares/inquirer.rb:8:in
>  `call'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/middlewares/geographer.rb:11:in
>  `call'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/middlewares/sphinxql.rb:14:in
>  `call'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/middleware-0.1.0/lib/middleware/runner.rb:31:in
>  `call'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/middleware-0.1.0/lib/middleware/builder.rb:102:in
>  `call'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search.rb:77:in
>  `populate'
> from 
> /home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search.rb:56:in
>  `meta'
> from 
> 

Re: [ts] Spelling suggestions for Thinking Sphinx 3

[Pat Allan]

Hi Miha

I’m really not familiar with other search tools, so I can’t offer any feedback 
on your second solution.

But, I would think it’d be reasonably easy to write your own code that uses 
raspell or a similar tool to get spelling suggestions on search queries, and 
then provide that information in your interface? That way, you can still use TS 
v3. This is probably the avenue I’d go down.

Cheers

— 
Pat

> On 14 Jul 2016, at 12:18 AM, Treeninja  wrote:
> 
> Hi!
> 
> I am using TS since version 2.0 and there I used Raspell to get some spelling 
> suggestions. Now we use version 3.0 and we lost that capability. Does anyone 
> has an alternative to the problem?
> 
> 1. Keep TS 3.0 with some other solution to achieve spelling suggestions.
> 2. Replace TS?
> 
> Any help will be greatly appreciated.
> 
> Kind regards, Miha
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] TS without model, but with index

[Pat Allan]

If you’re not using ActiveRecord models (at least, for the data you want to 
conduct searches on), then Thinking Sphinx isn’t the right fit. It is built to 
only work with ActiveRecord.

If you want to use Sphinx’s older binary protocol for searching, then the 
Riddle gem (which TS uses under the hood) can be used to construct search 
queries. Or if you’re using the mysql41 protocol and SphinxQL, then I’d 
recommend just using the mysql2 gem (which, again, is what TS does). I’d 
recommend the latter - there are features available in SphinxQL that aren’t in 
the older binary protocol.

If you want to generate your Sphinx configuration programmatically, then again, 
Riddle can do this too - but perhaps that’s overkill in your scenario?

Happy to go into detail on any of this if needed - but a good starting point 
for using mysql2 to conduct SphinxQL queries is the TS source code:
https://github.com/pat/thinking-sphinx/blob/develop/lib/thinking_sphinx/connection.rb#L99
 


Cheers

— 
Pat

> On 14 Jul 2016, at 7:24 AM, Юрий Кудряшов  wrote:
> 
> Hi, all.
> 
> I've  just started to setup TS and haven't found any information about next 
> steps in my situation.
> I've indexed lot's of data with xmlpipe2 (i don't use any databases, it's 
> just information from lots of files):
> 
> mysql> describe my_datas;
> +---++
> | Field | Type   |
> +---++
> | id| bigint |
> | name  | field  |
> | type  | field  |
> | name  | string |
> | is_closed | uint   |
> | cnt_count | uint   |
> +---++
> 
> How could i use and setup TS with rails?
> I just want to search information in index and display it to users.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Why is 2.0.6 required for TS 3?

[Pat Allan]

Hi Gabe

Thanks for the kind words, they’re very much appreciated :)

If it’s working fine for you, then I guess you can continue with Sphinx 2.0.4 - 
the requirement is due to the way the Riddle gem supports different Sphinx 
versions (TS is loading a version of Riddle that expects 2.0.6 or newer). It’s 
also worth noting that Sphinx 2.1.2 is recommended for TS v3.1.0 and onwards. 
So: testing locally and in a staging environment is recommended!

As an aside: Sphinx 2.0.6 was released in October 2012, and 2.1.2 in October 
2013 - 2.2.10 was released in September last year, and 2.3.1 as a beta in March 
this year. It may be worth hassling whoever maintains the sphinxsearch package 
for Ubuntu to get an update!

Cheers

— 
Pat

> On 11 Jul 2016, at 5:52 AM, Gabe da Silveira 
>  wrote:
> 
> Hi Pat,
> 
> First of all thanks for the years of service, you've been an amazing open 
> source citizen with the long term maintenance of TS, sometimes feels like a 
> rarity in the ruby world.
> 
> My question is why you recommend a minimum version specifically of 2.0.6 for 
> TS3?  The reason I ask is because the native Ubuntu 14.04 sphinxsearch 
> package is 2.0.4, and it seems to be working fine with TS 3.2.0.  In order to 
> update I need to get into PPAs and chef recipes and all that mess which I'd 
> rather avoid; unless of course there is good reason in which case I'll bite 
> the bullet.  
> 
> Thanks,
> -gabe
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Duplicate records returned

[Pat Allan]

You’re right, update_all doesn’t fire callbacks, and given you’re not using the 
status column in your index, that shouldn’t be a problem.

The duplicate records is an odd thing though. Are you finding _every_ result is 
duplicated? Only ones impacted by the cron job? Or are other records duplicated 
as well?

Also: which version of Thinking Sphinx are you using?

— 
Pat

> On 5 Jul 2016, at 9:20 PM, pratik ganvir <ganvirpra...@gmail.com> wrote:
> 
> Hi Pat,
> 
> Thanks for the reply. Here is my index definition.
> 
> ThinkingSphinx::Index.define :text_message, :with => :real_time do
>  has contact_group_id, type: :integer
>   has user_id, type: :integer
>   has id, :as => :sms_id, type: :integer
>   has sms_campeign_id, :as => :sms_campeign_id, type: :integer
>   has delivery_schedule, :type => :timestamp, :sortable => true
>   has created_at, :type => :timestamp, :sortable => true
> end
> 
> I am suing update_all method in my cron which surely doesn't fire callbacks.
> Here is the cron method that does that.
> 
> def self.update_system_errors
>   sms = Sms.where('status not in (?) and (created_at between ? 
> and ?) and job_id in 
> (?)',[10,11],DateTime.now.beginning_of_day,DateTime.now.end_of_day,["Daily 
> Message Limit Reached"])
> sms.update_all('status = 10')
> sms = Sms.where('status not in (?) and (created_at between ? and ?) and 
> job_id in 
> (?)',[10,11],DateTime.now.beginning_of_day,DateTime.now.end_of_day,["Sender 
> ID Does not Exist or Pending or Route Invalid!","No Sufficient Credits""])
> sms.update_all('status = 11')
> end
> 
> Thanks,
> Pratik Ganvir
> 
> On Sunday, 3 July 2016 16:46:43 UTC+5:30, Pat Allan wrote:
> Hi Pratik
> 
> Can you share your index definition? Also, how are you updating the 
> TextMessage records via your cron process? Is it in a manner that will fire 
> ActiveRecord callbacks? And is this the only model you have with a Sphinx 
> index?
> 
> Cheers
> 
> — 
> Pat
> 
>> On 2 Jul 2016, at 1:45 PM, pratik ganvir <ganvir...@gmail.com > 
>> wrote:
>> 
>> Hi,
>> 
>> So I have a model called TextMessage which has attributes id and user_id. I 
>> am using real time indices. 
>> There is one cron process that updates the status field of TextMessage 
>> records. When it does that, following query returns 2 records for each 
>> database record. 
>> 
>> TextMessage.search(with: {user_id: current_user.id 
>> <http://current_user.id/>},order: 'created_at desc',page: params[:page], 
>> per_page: 10 )
>> 
>> When I rebuild the index then everything goes fine. But considering the fact 
>> that I am using realtime indices I do not want to rebuild indexes everytime.
>> 
>> Please help me to get through this.
>> 
>> Thanks and regards,
>> Pratik
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphi...@googlegroups.com .
>> To post to this group, send email to thinkin...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] ThinkingSphinx::Configuration.instance.indices class loading funny business

[Pat Allan]

Hi Kyle

The defined indices are lazily loaded, so you’ll need to call preload_indices 
on the configuration instance before enumerating through each index. The error 
you’re seeing is from when - somewhere else in the TS code - preload_indices 
has been called, and thus all indices, including distributed indices are 
available. Distributed indices, by their nature, don’t have their own 
attributes, so you’ll want to ignore those.

One way of doing this:

  ThinkingSphinx::Configuration.instance.preload_indices
  index = ThinkingSphinx::Configuration.instance.indices.reject { |index|
index.distributed?
  }.detect { |index| index.model == klass }
  index.attributes.any? { |attribute| attribute.name == ‘updated_at’ }

Or, using a bit more of TS’s internals:

  index = ThinkingSphinx::IndexSet.new(classes: [klass]).to_a.first
  index.attributes.any? { |attribute| attribute.name == ‘updated_at’ }

Indices returned by the IndexSet class have distributed indices filtered out, 
and calls preload_indices for you.

Cheers

— 
Pat

> On 5 Jun 2016, at 10:38 AM, Kyle Ginavan  wrote:
> 
> Problem:
> In our application we also have a generic search function that uses many 
> different indices. Each index can be sorted differently. Therefore, we want 
> to examine the indices to determine if certain attributes are present and can 
> be sorted. We are attempting to use a line of code like:
> 
> ThinkingSphinx::Configuration.instance.indices.detect { |i| i.model == klass 
> }.attributes.any? {|a| a.name == "updated_at" }
> 
> Issue:
> The above works great most of the time. However, if we change the file where 
> this line of code exists more than 2 times (3 or more times) we receive the 
> below error:
> 
> NoMethodError - undefined method `attributes' for 
> #:
> 
> 
> 
> I assume there is some sort of Rails development class loading issue going on 
> here and I have no idea why it is the 3rd time where it fails. I'm unsure if 
> this same issue happens in production.
> 
> 
> 
> Question:
> 
> My question is #1 do you see a better way to solve this problem or #2 is this 
> a bug that needs logged?
> 
> 
> Stack:
> Rails: 4.2.5.1
> Ruby: 2.2.3
> OS: Linux & OSX
> Thinking Sphinx: 3.1.3
> Sphinx: 2.2.10
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] NEAR Operator with sentences wrapped in double quotes - doesn't seem to work.

[Pat Allan]

Hi Slava

I think the escaping is just an issue with the Rails logging side of things. 
I’ve tested queries directly with Sphinx, and through Rails, both with quotes, 
and they’re appearing the same in Sphinx’s logs:

# in a MySQL console connected to Sphinx:
mysql> SELECT * FROM spree_product_core WHERE MATCH('"apple"’);
# and in a Rails console:
> Spree::Product.search ‘"apple"'

The result in `log/development.searchd.query.log`:

[Mon May  9 21:11:36.231 2016] 0.001 sec 0.001 sec [ext2/1/ext 673 (0,20)] 
[spree_product_core] "apple"
[Mon May  9 21:13:01.752 2016] 0.001 sec 0.001 sec [ext2/0/ext 673 (0,20)] 
[spree_product_core] "apple"

And if I use NEAR, even though I get the escaping happening in the Rails 
log/console, I still get the expected results:

> Spree::Product.search ‘"granny smith" NEAR/2 "apple"'
  Sphinx Query (1.3ms)  SELECT * FROM `spree_product_core` WHERE 
MATCH('\"granny smith\" NEAR/2 \"apple\"') AND `sphinx_deleted` = 0 LIMIT 0, 20
  Sphinx  Found 61 results

I’m not sure the NEAR operator works like you’re expecting? Are you wanting 
exact matches of “by law”? Or are you looking for “law in law” or “in law law”?

— 
Pat

> On 6 May 2016, at 2:46 AM, Slava Yakobson  wrote:
> 
> 
> I am trying to use NEAR operator with two sentences - which means I need to 
> use double quotes for each sentence. I've tried multiple ways trying to get 
> it to work without luck.
> 
> 
> 1.
> q = '"law" NEAR/1 "by law"'
> Frame::Decision.search(q).count
> 
> Output: Sphinx Query (1.5ms)  SELECT * FROM `frame_decision_core` WHERE 
> MATCH('\"law\" NEAR/1 \"by law\"') AND `sphinx_deleted` = 0 LIMIT 0, 20
> 
> 2.
> q = %q("law" NEAR/1 "by law")
> Frame::Decision.search(Riddle::Query.escape(q)).count
> 
> Output:  Sphinx Query (1.1ms)  SELECT * FROM `frame_decision_core` WHERE 
> MATCH('\\\"law\\\" NEAR\\/1 \\\"by law\\\"') AND `sphinx_deleted` = 0 LIMIT 
> 0, 20
> 
> 3.
> q = '"law" NEAR/1 "by law"'
> Frame::Decision.search(q).count
> 
> Output: Sphinx Query (1.3ms)  SELECT * FROM `frame_decision_core` WHERE 
> MATCH('\"law\" NEAR/1 \"by law\"') AND `sphinx_deleted` = 0 LIMIT 0, 20
> 
> 
> 
> My guess is that using double quotes results in escape back slashes which 
> breaks things. The output probably has to look like this: MATCH('"law" NEAR/1 
> "by law"')
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] production.sphinx.conf does not exist

[Pat Allan]

When you say it only finds your development environment - what behaviour makes 
you say that? When you run `rake ts:index RAILS_ENV=production` does that 
generate a production configuration file and process the indices for it?

> On 10 Apr 2016, at 12:57 AM, Hou Fagon  wrote:
> 
> I am trying to deploy my app from development to production, after install 
> sphinx in server, run" rake ts:start RAILS_ENV=production" to make it work, 
> but it seems only find my development environment. I can't find  
> config/thinking_sphinx.yml as the tutorial 
> said:http://freelancing-gods.com/thinking-sphinx/deployment.html, anyone have 
> a idea how to generate the configure file and solve my problem? I must know 
> where to set the environment from develop to produciton? Thanks in advance!
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] No such filter attribute 'deleted_at'

[Pat Allan]

Also: if that doesn’t fix things, can you share the output of `rake ts:index` - 
thanks :)

> On 20 Mar 2016, at 3:53 PM, Pat Allan <p...@freelancing-gods.com> wrote:
> 
> Hi Shawn
> 
> Picking up from the discussion on Stack Overflow as well:
> http://stackoverflow.com/questions/35992666/no-such-filter-attribute-deleted-at-when-using-thinking-sphinx
>  
> <http://stackoverflow.com/questions/35992666/no-such-filter-attribute-deleted-at-when-using-thinking-sphinx>
> 
> Firstly: I’d recommend not adding an alias for the attribute unless there’s a 
> reason for it - so, just `has deleted_at` should be enough. And then, in 
> filters you definitely want to use :deleted_at everywhere, instead of having 
> the confusion of :deleted and :deleted_at both referring to the same value.
> 
> That said, I know you acknowledged that you’ve run `rake ts:rebuild` since 
> adding the attribute, but it doesn’t seem Sphinx has picked up the new 
> attribute. Can you run `rake ts:stop`, then check that the Sphinx daemon is 
> not running (`ps aux | grep searchd`). If there are still processes, kill 
> them manually, then run `rake ts:index ts:start` and see if that helps.
> 
> Cheers
> 
> — 
> Pat
> 
>> On 17 Mar 2016, at 3:53 AM, Shawn Dibble <shawndib...@gmail.com 
>> <mailto:shawndib...@gmail.com>> wrote:
>> 
>> I am experiencing some issues with Thinking Sphinx not allowing me to search 
>> by the deleted_at column in my database.
>> 
>> Essentially, I would like to get a list of all users, active users and 
>> deleted users, then be able to undelete certain users defined by the admin. 
>> Unfortunately, I can't even list the deleted users when using Thinking 
>> Sphinx. When I do a regular search for all users where 'deleted_at IS NOT 
>> NULL', I can get the users.
>> 
>> I can do any other field in my index just fine, but not deleted_at. Each 
>> time, I get this:
>> 
>> 
>> 
>> ERROR -- : index user_core: no such filter attribute 'deleted_at' 
>> (ThinkingSphinx::SphinxError)
>> 
>> This is my index file:
>> 
>> 
>> 
>> ThinkingSphinx::Index.define :user, :with => :active_record, :delta => true 
>> do 
>>   indexes first_name, :sortable => true 
>>   indexes last_name, :sortable => true 
>>   indexes email, :sortable => true 
>>   indexes phone, :sortable => true 
>>   has deleted_at, :as => :deleted 
>>   has id, :as => :user_id 
>>   has roles(:id), :as => :role_ids 
>>   set_property :delta => true 
>> end 
>> 
>> 
>> And my query:
>> 
>> params[:with][:deleted_at] = 0 
>> 
>> @results = User.search :conditions => params[:conditions], :with => 
>> params[:with], :order => params[:sort] + ' ' + params[:sort_mode], 
>> :page => params[:page], :per_page => params[:per_page], :star => true
>> 
>> NOTE: Even if I do just @results = User.search, I still get a list of only 
>> all active users where deleted_at is null. I do not seem to be able to get 
>> all users when using Thinking Sphinx. I have also tried 
>> params[:with][:deleted] with similar results.
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphinx+unsubscr...@googlegroups.com 
>> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
>> To post to this group, send email to thinking-sphinx@googlegroups.com 
>> <mailto:thinking-sphinx@googlegroups.com>.
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] No such filter attribute 'deleted_at'

[Pat Allan]

Hi Shawn

Picking up from the discussion on Stack Overflow as well:
http://stackoverflow.com/questions/35992666/no-such-filter-attribute-deleted-at-when-using-thinking-sphinx

Firstly: I’d recommend not adding an alias for the attribute unless there’s a 
reason for it - so, just `has deleted_at` should be enough. And then, in 
filters you definitely want to use :deleted_at everywhere, instead of having 
the confusion of :deleted and :deleted_at both referring to the same value.

That said, I know you acknowledged that you’ve run `rake ts:rebuild` since 
adding the attribute, but it doesn’t seem Sphinx has picked up the new 
attribute. Can you run `rake ts:stop`, then check that the Sphinx daemon is not 
running (`ps aux | grep searchd`). If there are still processes, kill them 
manually, then run `rake ts:index ts:start` and see if that helps.

Cheers

— 
Pat

> On 17 Mar 2016, at 3:53 AM, Shawn Dibble  wrote:
> 
> I am experiencing some issues with Thinking Sphinx not allowing me to search 
> by the deleted_at column in my database.
> 
> Essentially, I would like to get a list of all users, active users and 
> deleted users, then be able to undelete certain users defined by the admin. 
> Unfortunately, I can't even list the deleted users when using Thinking 
> Sphinx. When I do a regular search for all users where 'deleted_at IS NOT 
> NULL', I can get the users.
> 
> I can do any other field in my index just fine, but not deleted_at. Each 
> time, I get this:
> 
> 
> 
> ERROR -- : index user_core: no such filter attribute 'deleted_at' 
> (ThinkingSphinx::SphinxError)
> 
> This is my index file:
> 
> 
> 
> ThinkingSphinx::Index.define :user, :with => :active_record, :delta => true 
> do 
>   indexes first_name, :sortable => true 
>   indexes last_name, :sortable => true 
>   indexes email, :sortable => true 
>   indexes phone, :sortable => true 
>   has deleted_at, :as => :deleted 
>   has id, :as => :user_id 
>   has roles(:id), :as => :role_ids 
>   set_property :delta => true 
> end 
> 
> 
> And my query:
> 
> params[:with][:deleted_at] = 0 
> 
> @results = User.search :conditions => params[:conditions], :with => 
> params[:with], :order => params[:sort] + ' ' + params[:sort_mode], 
> :page => params[:page], :per_page => params[:per_page], :star => true
> 
> NOTE: Even if I do just @results = User.search, I still get a list of only 
> all active users where deleted_at is null. I do not seem to be able to get 
> all users when using Thinking Sphinx. I have also tried 
> params[:with][:deleted] with similar results.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Indexing fields that are null for ease of searching

[Pat Allan]

Hi Damon

To be honest, I’m not quite sure what the problem is - everything you’ve 
described thus far looks correct to me.

Perhaps one thing it’s worth trying is using an IF statement in your attribute 
definition, because MySQL’s boolean columns (boolean in Rails’ context, anyway) 
are integer values of 0 and 1, and that’s the same in Sphinx, so perhaps it’s 
worth returning an integer instead:

  has “IF(welcome_call_made IS NULL, 1, 0)”, :as => :need_welcome_call, :type 
=> :boolean

An alternative, though not as obvious when searching, is to have the column as 
a timestamp attribute, and NULL values will be stored as 0:

  has welcome_call_made

  Account.search :with => {:need_welcome_call => 0}

Hope this helps,

— 
Pat

> On 20 Feb 2016, at 4:29 AM, Damon Hill  wrote:
> 
> I have researched and read many posts on this and I thought I had found the 
> solution, but to no avail. 
> 
> I have several fields or attributes that can be null. A null value signifies 
> we need to perform some action on these entities, such as 
> "welcome_call_needed". Once the call is made, a timestamp for when the call 
> was made is added to the attribute. 
> 
> Here is my latest attempt at finding those attributes with null values. 
> 
> In my Account index file I have :
> 
> ThinkingSphinx::Index.define(:account,
> :with => :active_record,
> :delta => ThinkingSphinx::Deltas::DelayedDelta)  do
>   
>  
> 
>  has "welcome_call_made IS NULL", :as => :need_welcome_call, :type => 
> :boolean
> 
>  
> 
> end 
> 
> MySQL query as follows in the console: 
> 
> mysql> select count(*) from accounts WHERE welcome_call_made is NULL;
> +--+
> | count(*) |
> +--+
> | 3989 |
> +--+
> 1 row in set (0.00 sec)
> 
> 
> The query from the Rails 3.2.8 Console produces:
> 
> irb(main):005:0> accounts = Account.search :with => { :need_welcome_call => 
> true }
> => []
> 
> I am using Thinking Sphinx 3.1.4 and Ruby 1.9.3-p547 along with Rails 3.2.8 
> and MySQL 5.6. I am certain I am missing something very elementary. Sphinx is 
> phenomenal and if I can get passed this last small hurdle it will have 
> greatly improved the performance/response time in our app.
> 
> Thanks again for the assistance and the code.
> 
> Damon 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Facets sort problem

[Pat Allan]

Good to know you found the solution! I was certainly running out of ideas :)

> On 19 Feb 2016, at 11:03 PM, Treeninja <miha.jako...@gmail.com> wrote:
> 
> Hi!
> 
> Just to let you all know, I think I solved a problem. After installing a new 
> Sphinx, i had cleanup old stuff from the previous version: sudo service 
> sphinxsearch stop and remove files under the rails folder tmp\binlog, run 
> ts:configure, ts:index and ts:start and it worked.
> 
> Thanks!
> 
> Miha
> 
> On Thursday, February 18, 2016 at 1:57:01 PM UTC+1, Treeninja wrote:
> I only have ThinkingSphinx::SphinxQL.functions! specified under 
> thinking_sphinx.rb initializer.
> 
> Now I refreshed and restarted everything (no code changes) and i get this:
> 
> index product_core: sort-by attribute 'sphinx_internal_group' not found - 
> SELECT *, groupby()
> 
> On Thursday, February 18, 2016 at 1:51:29 PM UTC+1, Pat Allan wrote:
> Hmm. There’s nothing anywhere in your code referring to 
> ThinkingSphinx::SphinxQL? The variables! method on that module makes TS use 
> the old syntax (@count, @weight, etc), but it should default to the new 
> methods instead (count(), weight(), etc). The functions! method on that 
> module should switch things accordingly - but it shouldn’t need to be called! 
> TS 3.1.4 does that automatically.
> 
>> On 18 Feb 2016, at 11:36 PM, Treeninja <miha.j...@gmail.com <>> wrote:
>> 
>> OK.
>> 
>> This is now installed: Sphinx 2.2.10-id64-release (2c212e0)
>> 
>> I did not set  `ThinkingSphinx::SphinxQL.variables!`
>> 
>> @facets = Product.is 
>> <http://product.is/>_product_visible.is_created.facets @keyword_string, 
>> :order => "sphinx_internal_group ASC", :with => ts_filters
>> 
>> and I get:
>> 
>> Using the old-fashion @variables (@count, @weight, etc.) is deprecated - 
>> SELECT *, @groupby, @count FROM `product_core` 
>> 
>> I dont use @weight anymore...
>> 
>> On Thursday, February 18, 2016 at 1:14:03 PM UTC+1, Pat Allan wrote:
>> Ah, perhaps you’re using an old version of Sphinx, and calling 
>> `ThinkingSphinx::SphinxQL.variables!` somewhere? Try @groupby and @count 
>> instead.
>> 
>> Cheers
>> 
>> — 
>> Pat
>> 
>>> On 18 Feb 2016, at 10:55 PM, Treeninja <miha.j...@gmail.com <>> wrote:
>>> 
>>> Hi Pat!
>>> 
>>> Thank you for your answer. Now I get
>>> 
>>> index product_core: sort-by attribute 'sphinx_internal_group' not found
>>> 
>>> Should I add something to index definition?
>>> 
>>> Regards, Miha
>>> 
>>> On Thursday, February 18, 2016 at 12:33:33 PM UTC+1, Pat Allan wrote:
>>> Hi Miha
>>> 
>>> If you want all facets sorted the same way, then you can use the :order 
>>> option and sort either by sphinx_internal_group (the facet key values) or 
>>> sphinx_internal_count (the number of results for a facet key) ASC/DESC.
>>> 
>>>   Product.is <http://product.is/>_product_visible.is_created.facets 
>>> @keyword_string, :order => ‘sphinx_internal_group ASC’, :with => ts_filters
>>> 
>>> However, if you only want some facets sorted a specific way, then I’m 
>>> afraid you’re best served by assigning the sorted values (as you’re 
>>> computing in your example) as new objects, instead of trying to modify the 
>>> TS facet search results.
>>> 
>>> Cheers
>>> 
>>> — 
>>> Pat
>>> 
>>>> On 18 Feb 2016, at 10:15 PM, Treeninja <miha.j...@gmail.com <>> wrote:
>>>> 
>>>> Hi!
>>>> 
>>>> I've migrated to TS 3.1.4 and have problem with facets sorting.
>>>> 
>>>> I am using this:
>>>> 
>>>> @facets = Product.is 
>>>> <http://product.is/>_product_visible.is_created.facets @keyword_string, 
>>>> :order => (order), :with => ts_filters
>>>> @facets[:industry_id] = Hash[@facets[:industry_id].sort]
>>>> @facets[:country_id]= Hash[@facets[:country_id].sort]
>>>> 
>>>> Error is: 
>>>> 
>>>> undefined method `[]=' for #
>>>> 
>>>> How can i properly sort facets?
>>>> 
>>>> Thanks.
>>>> 
>>>> Best regards, Miha
>>>> 
>>>> -- 
>>>> You received this message because you are subscribed to the Google Groups 
>>>> "Thinking Sphinx" group.
>>>> To unsubscrib

Re: [ts] Facets sort problem

[Pat Allan]

Hmm. There’s nothing anywhere in your code referring to 
ThinkingSphinx::SphinxQL? The variables! method on that module makes TS use the 
old syntax (@count, @weight, etc), but it should default to the new methods 
instead (count(), weight(), etc). The functions! method on that module should 
switch things accordingly - but it shouldn’t need to be called! TS 3.1.4 does 
that automatically.

> On 18 Feb 2016, at 11:36 PM, Treeninja <miha.jako...@gmail.com> wrote:
> 
> OK.
> 
> This is now installed: Sphinx 2.2.10-id64-release (2c212e0)
> 
> I did not set  `ThinkingSphinx::SphinxQL.variables!`
> 
> @facets = Product.is_product_visible.is_created.facets @keyword_string, 
> :order => "sphinx_internal_group ASC", :with => ts_filters
> 
> and I get:
> 
> Using the old-fashion @variables (@count, @weight, etc.) is deprecated - 
> SELECT *, @groupby, @count FROM `product_core` 
> 
> I dont use @weight anymore...
> 
> On Thursday, February 18, 2016 at 1:14:03 PM UTC+1, Pat Allan wrote:
> Ah, perhaps you’re using an old version of Sphinx, and calling 
> `ThinkingSphinx::SphinxQL.variables!` somewhere? Try @groupby and @count 
> instead.
> 
> Cheers
> 
> — 
> Pat
> 
>> On 18 Feb 2016, at 10:55 PM, Treeninja <miha.j...@gmail.com > 
>> wrote:
>> 
>> Hi Pat!
>> 
>> Thank you for your answer. Now I get
>> 
>> index product_core: sort-by attribute 'sphinx_internal_group' not found
>> 
>> Should I add something to index definition?
>> 
>> Regards, Miha
>> 
>> On Thursday, February 18, 2016 at 12:33:33 PM UTC+1, Pat Allan wrote:
>> Hi Miha
>> 
>> If you want all facets sorted the same way, then you can use the :order 
>> option and sort either by sphinx_internal_group (the facet key values) or 
>> sphinx_internal_count (the number of results for a facet key) ASC/DESC.
>> 
>>   Product.is <http://product.is/>_product_visible.is_created.facets 
>> @keyword_string, :order => ‘sphinx_internal_group ASC’, :with => ts_filters
>> 
>> However, if you only want some facets sorted a specific way, then I’m afraid 
>> you’re best served by assigning the sorted values (as you’re computing in 
>> your example) as new objects, instead of trying to modify the TS facet 
>> search results.
>> 
>> Cheers
>> 
>> — 
>> Pat
>> 
>>> On 18 Feb 2016, at 10:15 PM, Treeninja <miha.j...@gmail.com <>> wrote:
>>> 
>>> Hi!
>>> 
>>> I've migrated to TS 3.1.4 and have problem with facets sorting.
>>> 
>>> I am using this:
>>> 
>>> @facets = Product.is 
>>> <http://product.is/>_product_visible.is_created.facets @keyword_string, 
>>> :order => (order), :with => ts_filters
>>> @facets[:industry_id] = Hash[@facets[:industry_id].sort]
>>> @facets[:country_id]= Hash[@facets[:country_id].sort]
>>> 
>>> Error is: 
>>> 
>>> undefined method `[]=' for #
>>> 
>>> How can i properly sort facets?
>>> 
>>> Thanks.
>>> 
>>> Best regards, Miha
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Thinking Sphinx" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to thinking-sphi...@googlegroups.com <>.
>>> To post to this group, send email to thinkin...@googlegroups.com <>.
>>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>>> <https://groups.google.com/group/thinking-sphinx>.
>>> For more options, visit https://groups.google.com/d/optout 
>>> <https://groups.google.com/d/optout>.
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphi...@googlegroups.com .
>> To post to this group, send email to thinkin...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubs

Re: [ts] Facets sort problem

[Pat Allan]

Ah, perhaps you’re using an old version of Sphinx, and calling 
`ThinkingSphinx::SphinxQL.variables!` somewhere? Try @groupby and @count 
instead.

Cheers

— 
Pat

> On 18 Feb 2016, at 10:55 PM, Treeninja <miha.jako...@gmail.com> wrote:
> 
> Hi Pat!
> 
> Thank you for your answer. Now I get
> 
> index product_core: sort-by attribute 'sphinx_internal_group' not found
> 
> Should I add something to index definition?
> 
> Regards, Miha
> 
> On Thursday, February 18, 2016 at 12:33:33 PM UTC+1, Pat Allan wrote:
> Hi Miha
> 
> If you want all facets sorted the same way, then you can use the :order 
> option and sort either by sphinx_internal_group (the facet key values) or 
> sphinx_internal_count (the number of results for a facet key) ASC/DESC.
> 
>   Product.is <http://product.is/>_product_visible.is_created.facets 
> @keyword_string, :order => ‘sphinx_internal_group ASC’, :with => ts_filters
> 
> However, if you only want some facets sorted a specific way, then I’m afraid 
> you’re best served by assigning the sorted values (as you’re computing in 
> your example) as new objects, instead of trying to modify the TS facet search 
> results.
> 
> Cheers
> 
> — 
> Pat
> 
>> On 18 Feb 2016, at 10:15 PM, Treeninja <miha.j...@gmail.com > 
>> wrote:
>> 
>> Hi!
>> 
>> I've migrated to TS 3.1.4 and have problem with facets sorting.
>> 
>> I am using this:
>> 
>> @facets = Product.is 
>> <http://product.is/>_product_visible.is_created.facets @keyword_string, 
>> :order => (order), :with => ts_filters
>> @facets[:industry_id] = Hash[@facets[:industry_id].sort]
>> @facets[:country_id]= Hash[@facets[:country_id].sort]
>> 
>> Error is: 
>> 
>> undefined method `[]=' for #
>> 
>> How can i properly sort facets?
>> 
>> Thanks.
>> 
>> Best regards, Miha
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphi...@googlegroups.com .
>> To post to this group, send email to thinkin...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/thinking-sphinx 
>> <https://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Facets sort problem

[Pat Allan]

Hi Miha

If you want all facets sorted the same way, then you can use the :order option 
and sort either by sphinx_internal_group (the facet key values) or 
sphinx_internal_count (the number of results for a facet key) ASC/DESC.

  Product.is_product_visible.is_created.facets @keyword_string, :order => 
‘sphinx_internal_group ASC’, :with => ts_filters

However, if you only want some facets sorted a specific way, then I’m afraid 
you’re best served by assigning the sorted values (as you’re computing in your 
example) as new objects, instead of trying to modify the TS facet search 
results.

Cheers

— 
Pat

> On 18 Feb 2016, at 10:15 PM, Treeninja  wrote:
> 
> Hi!
> 
> I've migrated to TS 3.1.4 and have problem with facets sorting.
> 
> I am using this:
> 
> @facets = Product.is_product_visible.is_created.facets @keyword_string, 
> :order => (order), :with => ts_filters
> @facets[:industry_id] = Hash[@facets[:industry_id].sort]
> @facets[:country_id]= Hash[@facets[:country_id].sort]
> 
> Error is: 
> 
> undefined method `[]=' for #
> 
> How can i properly sort facets?
> 
> Thanks.
> 
> Best regards, Miha
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] In thinking sphinx min_infix_len not working in production

[Pat Allan]

Two things it’s worth checking: first, are you using the same version of Sphinx 
in development and production?

And, with older versions of Sphinx, it might be better to add enable_star: true 
to each environment in your config/sphinx.yml file, and then search using :star 
=> true (or add the wildcards yourself: 123*

Hope this helps!

— 
Pat

> On 10 Feb 2016, at 9:28 PM, abhi...@railsfactory.org wrote:
> 
> I have used min_infix_len => 2 in my model.rb file to improve searching of 
> items by splitting up the words in sphinx. Everything is working fine in 
> development.
> 
> But in my production the words are not getting split up and searched.
> 
> 
> 
> For example,
> 
> If I have string like this "1234567890". I am getting the string in search 
> result if I give the entire string as "1234567890". But if i give as "123" 
> the search term is not getting searched and shown. To overcome this problem I 
> have use min_infix_len => 2 in model.rb like below.
> 
>  define_index do
>   indexes id
>   indexes title
>   indexes ticket_status
>   indexes state
>   set_property :min_infix_len => 3
>  end
> 
> 
> And generated the index, this get updated in my development.sphinx.conf as 
> below:
> 
> index ticket_core
> {
>   path = 
> /home/abhiram/projects/railsfactory/connect/db/sphinx/development/ticket_c   
>  index ticket_core
>charset_type = utf-8
>min_infix_len = 3
>source = ticket_core_0
>  }
> 
> 
> After this if restart sphinx and search I got the expected result and that 
> string is getting searched as I expected.
> 
> But in production I have the same setup, index also getting updated in my 
> production.sphinx.conf
> 
>  index ticket_core
>charset_type = utf-8
>min_infix_len = 3
>source = ticket_core_0
>  }
> 
> 
> After restarting sphinx, normal search is working fine but the substring 
> search which I expected like above is not working like it worked in 
> development. The string is not getting searched without giving the complete 
> string in the field.
> 
> This is my sphinx.yml
> 
> development:
>   bin_path: '/usr/bin'
> production:
>   bin_path: '/usr/bin'
> 
> 
> I don't no how to debug this in sphinx. Can any body guide me in this. I want 
> to know how to make min_infix_len = 3 to work in production.
> 
> 
> 
> I am using old version of rails, since the app is old:
> 
> 
> 
> rails 2.1 ruby 1.8.7 thinking-sphinx 1.4.14
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Re: searching for empty fields

[Pat Allan]

Hi Yuri

What you’re doing looks correct - which version of Thinking Sphinx are you 
using?

— 
Pat

> On 27 Jan 2016, at 8:57 PM, yuri.redae...@vidiemme.it wrote:
> 
> Hi,
> 
> I'm trying to do the same thing but still have an empty result
> 
> In my case:
> 
> has "LENGTH(has_countries) < 0", :as => :empty_countries, :type => :boolean
> 
> result = Digitalasset.search match_mode: :extended, :with => 
> {:empty_countries => true}, page: page, per_page: per_page, :order => 
> :updated_at, :sort_mode => :desc
> 
> Thanks in advance and sorry for my english.
> 
> Il giorno giovedì 13 gennaio 2011 21:28:12 UTC+1, Simon ha scritto:
> Clever.  Thanks for the help! 
> 
> Simon 
> 
> On Jan 13, 2:33 am, Pat Allan <p...@freelancing-gods.com <>> wrote: 
> > Hi Simon 
> > 
> > Sphinx has no concept of searching for empty fields. I think a better 
> > option would be to add an attribute indicating whether the specific column 
> > is blank: 
> > 
> >   has "LENGTH(body) < 0", :as => :empty_body, :type => :boolean 
> > 
> > That should work for MySQL - for PostgreSQL, I think the function is 
> > char_length. 
> > 
> > And then in searches, you can do the following: 
> > 
> >   Article.search :with => {:empty_body => true} 
> > 
> > Add another attribute for the title as well, and then you can use that 
> > filter. 
> > 
> > Hope this helps. 
> > 
> > -- 
> > Pat 
> > 
> > On 11/01/2011, at 1:10 PM, Simon wrote: 
> > 
> > > Hi there, 
> > 
> > > I am trying to search for empty strings, and am getting back no 
> > > results.  I am indexing two values, title and body, in my define_index 
> > > block.  I am then trying to something along the lines of 
> > > Article.search :conditions => {:body => ""} but I get back no 
> > > results.  I know there are indexed entries with blank bodies, so I 
> > > feel I must be missing something here. 
> > 
> > > If this is possible and I get it going, I would also like to be able 
> > > to search for Articles with both blank bodies and titles.  Can I just 
> > > add both fields into the conditions hash?  Will this get treated as an 
> > > AND condition? 
> > 
> > > Thanks, 
> > 
> > > Simon 
> > 
> > > -- 
> > > You received this message because you are subscribed to the Google Groups 
> > > "Thinking Sphinx" group. 
> > > To post to this group, send email to thinkin...@ <>googlegroups. 
> > > <http://googlegroups.com/>com <http://googlegroups.com/>. 
> > > To unsubscribe from this group, send email to thinking-sphi...@ 
> > > <>googlegroups.com <http://googlegroups.com/>. 
> > > For more options, visit this group 
> > > athttp://groups.google.com/group/thinking-sphinx?hl=en 
> > > <http://groups.google.com/group/thinking-sphinx?hl=en>.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> <https://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.


-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Triggering delta indexing

[Pat Allan]

Nope, that’s only for TS v1/v2. In recent versions of v3, you’d do the 
following:

ThinkingSphinx::Deltas::IndexJob.new(‘forum_post_delta').perform

The argument is the name of the index (including the _delta suffix), not the 
model.

Cheers

— 
Pat

> On 13 Jan 2016, at 11:38 AM, Martin Sommer  wrote:
> 
> Pat, is this still a valid call to trigger delta indexing?
> 
> ForumPost.index_delta
> 
> Thanks,
> Martin
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] ThinkingSphinx.search_for_ids... any way to get the class those ids belong to

[Pat Allan]

Hi Kyle

Try the following instead:

ThinkingSphinx.search(
  ‘Iverson’,
  :middleware => ThinkingSphinx::Middlewares::RAW_ONLY,
  :classes=> [Contact, Contract, License]
)

Cheers

— 
Pat

> On 23 Dec 2015, at 9:10 am, Kyle Ginavan <kylejgina...@gmail.com> wrote:
> 
> I see *.results is not longer supported. I'm trying to do something very 
> similar where I get the ids and class associated to each id:
> 
> ThinkingSphinx.search_for_ids( "Iverson", {:classes => [Contact, Contract, 
> License]} ).results
> 
> What is the current way to accomplish this?
> 
> Cheers!
> 
> On Friday, October 30, 2009 at 10:54:41 PM UTC-5, Pat Allan wrote:
> You should be able to access the raw Sphinx results via the following: 
> 
>ids = ThinkingSphinx.search_for_ids 'Foo', 
>  :classes => [Bar, Baz, Doom] 
>ids.results 
> 
> In ids.results[:matches], you'll find the attributes hash, which   
> contains a class_crc value, which matches each model name's crc code   
> (ie: Bar.to_crc32). 
> 
> It's not neat, but that's the only way at the moment. 
> 
> -- 
> Pat 
> 
> On 29/10/2009, at 2:19 AM, halogen64 wrote: 
> 
> > 
> > ThinkingSphinx.search_for_ids('Foo', :classes => [Bar, Baz, Doom]) 
> > 
> > only gives me 
> > 
> > [1, 2, 4] 
> > 
> > For which I can do nothing with, is there anyway to get what class 
> > those ids belong to? 
> > 
> > > 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com.
> To post to this group, send email to thinking-sphinx@googlegroups.com.
> Visit this group at https://groups.google.com/group/thinking-sphinx.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Thinking Sphinx Error

[Pat Allan]

… sorry, I meant the jdbc-mysql library, not ruby-mysql. Which does raise the 
question: which version of jdbc-mysql are you using?

> On 18 Nov 2015, at 4:17 pm, Pat Allan <p...@freelancing-gods.com> wrote:
> 
> Hi Suresh
> 
> I think this is related to JRuby (perhaps the ruby-mysql library)… one thing 
> that may help is adding `mysql_version_string: 5.5.21` to each environment in 
> `config/thinking_sphinx.yml` (you’ll need to create the file if you don’t 
> have it already):
> 
>   development:
> mysql_version_string = 5.5.21
>   test:
> mysql_version_string = 5.5.21
>   production:
> mysql_version_string = 5.5.21
> 
> Once you’ve made this change, run `rake ts:rebuild`.
> 
> If that doesn’t help, can you let us know which version of Sphinx you’re 
> using?
> 
> Cheers
> 
> — 
> Pat
> 
>> On 18 Nov 2015, at 3:59 pm, sur...@mahaswami.com 
>> <mailto:sur...@mahaswami.com> wrote:
>> 
>> Hi,
>> 
>> I am new to thinking sphinx.
>> 
>> I am using torquebox  server version 3.1.2 and i am running it in jruby mode 
>> and jruby version is 1.7.13 to run my application.
>> 
>> Thinking Sphinx version is 3.1.4 and i don't have any issues to run the 
>> sphinx server.
>> 
>> Whenever i search the keyword thinking sphinx giving the following error.
>> 
>> ThinkingSphinx::SyntaxError (sphinxql: syntax error, unexpected AS, 
>> expecting $end near 'AS auto_increment_increment, @@character_set_client AS 
>> character_set_client, @@character_set_connection AS 
>> character_set_connection, @@character_set_results AS character_set_results, 
>> @@character_set_server AS character_set_server, @@init_connect AS 
>> init_connect, @@interactive_timeout AS interactive_timeout, @@license AS 
>> license, @@lower_case_table_names AS lower_case_table_names, 
>> @@max_allowed_packet AS max_allowed_packet, @@net_buffer_length AS 
>> net_buffer_length, @@net_write_timeout AS net_write_timeout, 
>> @@query_cache_size AS query_cache_size, @@query_cache_type AS 
>> query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS 
>> system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS tx_isolation, 
>> @@wait_timeout AS wait_timeout' - SELECT * FROM `community_core`, 
>> `community_delta`, `cg_community_profile_core`, `cg_community_profile_delta` 
>> WHERE MATCH('user') AND `profile_id` IN (79) AND `sphinx_deleted` = 0 LIMIT 
>> 0, 20 OPTION field_weights=(first_name=150, last_name=150, 
>> community_name=10); SHOW META):
>>   
>> /usr/local/rvm/gems/jruby-1.7.13/bundler/gems/thinking-sphinx-c8f549f68907/lib/thinking_sphinx/connection.rb:93:in
>>  `query'
>>   
>> /usr/local/rvm/gems/jruby-1.7.13/bundler/gems/thinking-sphinx-c8f549f68907/lib/thinking_sphinx/connection.rb:77:in
>>  `query_all'
>>   
>> /usr/local/rvm/gems/jruby-1.7.13/bundler/gems/thinking-sphinx-c8f549f68907/lib/thinking_sphinx/search/batch_inquirer.rb:17:in
>>  `results'
>>   
>> /usr/local/rvm/gems/jruby-1.7.13/bundler/gems/thinking-sphinx-c8f549f68907/lib/thinking_sphinx/connection.rb:39:in
>>  `take'
>>   innertube (1.1.0) lib/innertube.rb:138:in `take'
>> 
>> Could anyone give the reason for this error?
>> 
>> Regards,
>> Suresh.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Thinking Sphinx" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to thinking-sphinx+unsubscr...@googlegroups.com 
>> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
>> To post to this group, send email to thinking-sphinx@googlegroups.com 
>> <mailto:thinking-sphinx@googlegroups.com>.
>> Visit this group at http://groups.google.com/group/thinking-sphinx 
>> <http://groups.google.com/group/thinking-sphinx>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> <mailto:thinking-sphinx+unsubscr...@googlegroups.com>.
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> <mailto:thinking-sphinx@googlegroups.com>.
> Visit this group at http://groups.google.com/group/thinking-sphinx 
> <http://groups.google.com/group/thinking-sphinx>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at http://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Thinking Sphinx Error

[Pat Allan]

Hi Suresh

I think this is related to JRuby (perhaps the ruby-mysql library)… one thing 
that may help is adding `mysql_version_string: 5.5.21` to each environment in 
`config/thinking_sphinx.yml` (you’ll need to create the file if you don’t have 
it already):

  development:
mysql_version_string = 5.5.21
  test:
mysql_version_string = 5.5.21
  production:
mysql_version_string = 5.5.21

Once you’ve made this change, run `rake ts:rebuild`.

If that doesn’t help, can you let us know which version of Sphinx you’re using?

Cheers

— 
Pat

> On 18 Nov 2015, at 3:59 pm, sur...@mahaswami.com wrote:
> 
> Hi,
> 
> I am new to thinking sphinx.
> 
> I am using torquebox  server version 3.1.2 and i am running it in jruby mode 
> and jruby version is 1.7.13 to run my application.
> 
> Thinking Sphinx version is 3.1.4 and i don't have any issues to run the 
> sphinx server.
> 
> Whenever i search the keyword thinking sphinx giving the following error.
> 
> ThinkingSphinx::SyntaxError (sphinxql: syntax error, unexpected AS, expecting 
> $end near 'AS auto_increment_increment, @@character_set_client AS 
> character_set_client, @@character_set_connection AS character_set_connection, 
> @@character_set_results AS character_set_results, @@character_set_server AS 
> character_set_server, @@init_connect AS init_connect, @@interactive_timeout 
> AS interactive_timeout, @@license AS license, @@lower_case_table_names AS 
> lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, 
> @@net_buffer_length AS net_buffer_length, @@net_write_timeout AS 
> net_write_timeout, @@query_cache_size AS query_cache_size, @@query_cache_type 
> AS query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS 
> system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS tx_isolation, 
> @@wait_timeout AS wait_timeout' - SELECT * FROM `community_core`, 
> `community_delta`, `cg_community_profile_core`, `cg_community_profile_delta` 
> WHERE MATCH('user') AND `profile_id` IN (79) AND `sphinx_deleted` = 0 LIMIT 
> 0, 20 OPTION field_weights=(first_name=150, last_name=150, 
> community_name=10); SHOW META):
>   
> /usr/local/rvm/gems/jruby-1.7.13/bundler/gems/thinking-sphinx-c8f549f68907/lib/thinking_sphinx/connection.rb:93:in
>  `query'
>   
> /usr/local/rvm/gems/jruby-1.7.13/bundler/gems/thinking-sphinx-c8f549f68907/lib/thinking_sphinx/connection.rb:77:in
>  `query_all'
>   
> /usr/local/rvm/gems/jruby-1.7.13/bundler/gems/thinking-sphinx-c8f549f68907/lib/thinking_sphinx/search/batch_inquirer.rb:17:in
>  `results'
>   
> /usr/local/rvm/gems/jruby-1.7.13/bundler/gems/thinking-sphinx-c8f549f68907/lib/thinking_sphinx/connection.rb:39:in
>  `take'
>   innertube (1.1.0) lib/innertube.rb:138:in `take'
> 
> Could anyone give the reason for this error?
> 
> Regards,
> Suresh.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at http://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at http://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Eager Loading in TS

[Pat Allan]

Very sorry for not responding sooner - the past few months have been filled 
with travel, and my inbox has suffered.

Perhaps you don’t need these answers any more, but just in case (or for anyone 
else who comes across this thread):

* Using GET requests for search actions is highly recommended. If you were 
modifying data, then a POST, PUT, PATCH or DELETE would be appropriate, but if 
the data’s not changing, then always opt for a GET.

* The documentation is available here:
http://freelancing-gods.com/thinking-sphinx/ 

And specifically, for the :sql/:include options:
http://freelancing-gods.com/thinking-sphinx/searching.html#advanced 


Again, sorry for the delay.

— 
Pat

> On 16 Sep 2015, at 10:45 pm, Игорь Калашнюк  wrote:
> 
> Of course that helped a lot.
> Thanks.
> And can you tell me one more thing. This time about pagination.
> I beg a pardon for my english :)
> 
> On my page I've had such search form
> 
> = form_tag({ controller: 'board',
>  action: 'all',
>  id: params[:id]},
>  class: 'navbar-form navbar-left') do |f|
> 
> the controller is in my previous post.
> 
> So the pagination for searching results didn't worked
> till I've changed http method to GET ( i did it becouse i wanted to have 
> searching results in browser history )
> 
> Is such behavior is ok ?
> 
> Now I have both ( address and pagination )
> here is my new form ( don't pay atention to action: and controller: )
> = form_tag( url_for_options = { controller: 'board',
> action: 'all',
> id: params[:id]},
> options = { method: :get,
> class: 'navbar-form navbar-left'}) do |f|
> 
>  And one more thing.
> Where can i find documentation for Thinking Sphinx v3 ?
> I've was searching a lot but this construction ( :sql => {:include => 
> :vk_photos},) i see for the first ?
> Thanks for your help.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at http://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at http://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Negative weights

[Pat Allan]

Hi Adam

I’ve not come across negative weights, but it’s fair to say you’re using more 
complex queries than I generally deal with in my own projects. It’s probably 
worth asking these questions on the Sphinx forum to get a better answer than 
mine :)
http://sphinxsearch.com/forum/ 

— 
Pat

> On 4 Nov 2015, at 7:28 am, Adam Gamble  wrote:
> 
> So when a user inputs a query like "dog" we send something like this to sphinx
> 
> dog | "cat" | "dog and cat" | "puppy" | "pets" | "animals" | "dog silhouette" 
> | "horse" | "dog isolated" | "walking dog"
> 
> 
> 
> the problem here is that sphinx gives the same weight to dog as it does "cat" 
> or "dog and cat" and we don't want that. so my idea first was to boost the 
> user query like so
> 
> 
> dog^2 | "cat" | "dog and cat" | "puppy" | "pets" | "animals" | "dog 
> silhouette" | "horse" | "dog isolated" | "walking dog"
> 
> 
> 
> Which kinda works, but not perfectly. So I tried
> 
> 
> dog | "cat^-.25" | "dog and cat^-.25" | "puppy^-.25" | "pets^-.25" | 
> "animals^-.25" | "dog silhouette^-.25" | "horse^-.25" | "dog isolated^-.25" | 
> "walking dog^-.25"
> 
> 
> and this makes the user query the most relevant, but seems to basically 
> blocked out related queries. I'm trying to have "dog" results be the most 
> relevant with related terms (the or queries) fall in behind that.
> 
> Is there a canonical way to do something like this? Also is negative query 
> weights even a supported thing?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at http://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at http://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.

Re: [ts] Indexer conditions

[Pat Allan]

Hi Miha

The best approach is to add an association to your Company model for just 
default sites:

  has_many :default_sites, :class_name => ‘Site’, :conditions => {:is_default 
=> true}

And then the attribute just changes to reference that:

  has default_sites.country_id, :facet => true

Granted, the conditions in the association is deprecated (given you’re using TS 
v2, I’m guessing you’re still on Rails 3). An alternative that is friendlier 
across Rails/TS versions could be to add a WHERE clause within the index 
definition:

  # use original attribute definition:
  has sites.country_id, :facet => true

  # For PostgreSQL:
  where “sites.is_default IS NULL OR sites.is_default = TRUE”
  # Or, for MySQL:
  where “sites.is_default IS NULL OR sites.is_default = 1”

This is not tested though, and it’s worth double-checking to confirm the right 
results are being returned. My thinking is to have the NULL check because some 
companies may not have any default sites, and you still want those companies 
being part of the Sphinx data. There may be a better way of doing this…

Hope this is helpful.

— 
Pat

> On 29 Oct 2015, at 7:59 am, Treeninja  wrote:
> 
> Hi!
> 
> I am using TS 2 and would like to set some conditions on certain fields in 
> the index.
> 
> A company has many sites, which contain country_id. So the index inside 
> Company model is:
> 
>  define_index do
> indexes :name, :sortable => true
> indexes :description
> 
> has sites.country_id, :facet => true
> end
> 
> The sites table has is_default boolean value and I would like to only index 
> sites that has is_default = true.
> 
> How can I do that in the index?
> 
> Kind regards, Miha
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to thinking-sphinx+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to thinking-sphinx@googlegroups.com 
> .
> Visit this group at http://groups.google.com/group/thinking-sphinx 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to thinking-sphinx+unsubscr...@googlegroups.com.
To post to this group, send email to thinking-sphinx@googlegroups.com.
Visit this group at http://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.