Heya Mike, This sounds like an odd problem indeed.
Firstly: I think UTF8 is the way to go for the database - and indeed, I’d expect it to be the same for Sphinx/TS (unless you’ve configured it to be something else?). Can you confirm that delta indexing is occurring when a record with Polish characters is updated? (I presume it’s being done in a standard way that’s firing validations and callbacks) The fact things aren’t coming through in excerpts is surprising as well… I’m not yet sure what the cause of that might be. Can you confirm if you’ve any character/encoding related settings in `config/thinking_sphinx.yml`? Also, it may be worth upgrading to Sphinx 2.2.11 - it seems there’s one documented fix in that for excerpts and UTF8 (when using wildcards). Cheers, — Pat > On 6 Sep 2019, at 11:44 pm, mikej <[email protected]> wrote: > > A client recently was unable to save a record as it contained Polish > characters. In order to accommodate these records (rightly or wrongly) I > changed the db collation with a migration (in development only so far). > First for the whole db, then by table. i.e. > > class ChangeEncoding < ActiveRecord::Migration[5.2] > def change > config = Rails.configuration.database_configuration > db_name = config[Rails.env]["database"] > collate = 'utf8_general_ci' > char_set = 'utf8' > execute("ALTER DATABASE #{db_name} CHARACTER SET #{char_set} COLLATE > #{collate};") > ActiveRecord::Base.connection.tables.each do |table| > execute("ALTER TABLE #{table} CONVERT TO CHARACTER SET #{char_set} > COLLATE #{collate};") > end > end > end > > Records can now be saved. > > I am using delta indexing. If you search for a record containing the Polish > characters, no records are found until issuing rake ts:index. Once a record > is found, it is not included in the excerpts result. All other characters > are returned within span.match. > > Any ideas on how I can configure sphinx to find records without indexing and > include all characters in the excerpt? > > Many thanks, > > Mike > > Sphinx 2.2.9 > thinking-sphinx 4.2.0 > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/thinking-sphinx/f58b6b0d-52b2-4afe-853b-0bfe6fe2babe%40googlegroups.com > > <https://groups.google.com/d/msgid/thinking-sphinx/f58b6b0d-52b2-4afe-853b-0bfe6fe2babe%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/thinking-sphinx/80E38254-75D7-4371-A8F6-CCB9366D0EB5%40freelancing-gods.com.
