Re: [rt-users] docs improvement suggestion for full-text searching
On 12/08/2014 01:12 PM, Jo Rhett wrote: Sphinx refuses to run without that parameter. Which given that it wasn’t defined in the file they really should have set the default appropriately and not whined at the user, but this is the version of Sphinx currently in RHEL EPEL so there’s going to be a lot of RHEL/CentOS users running into this problem. On Dec 8, 2014, at 11:24 AM, Alex Vandiver ale...@bestpractical.com wrote: I can't replicate the compat_sphinxql_magics problems you report with a stock Sphinx 2.0.8 from EPEL on CentOS 6. With a stock configuration as provided by 4.2-trunk, indexer runs with no errors (see below). If it RT's use of sphinx requires the daemon to answer, which you didn’t start. With your example file when I start the daemon I get this error: # service searchd start Starting searchd: Sphinx 2.0.8-id64-release (r3831) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/sphinx/sphinx.conf'... WARNING: compat_sphinxql_magics=1 is deprecated; please update your application and config Note that your file doesn’t contain magics=1. So I totally agree that it’s an annoying bug that the developers should fix, but it’s also something you can avoid by putting it in the file you output. -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On 12/14/2014 08:25 PM, Jo Rhett wrote: RT's use of sphinx requires the daemon to answer, which you didn’t start. With your example file when I start the daemon I get this error: # service searchd start Starting searchd: Sphinx 2.0.8-id64-release (r3831) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/sphinx/sphinx.conf'... WARNING: compat_sphinxql_magics=1 is deprecated; please update your application and config As the output notes, that is merely a warning. It is not an error. searchd starts up just fine for me: # service searchd start Starting searchd: Sphinx 2.0.8-id64-release (r3831) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/etc/sphinx/sphinx.conf'... WARNING: compat_sphinxql_magics=1 is deprecated; please update your application and config listening on all interfaces, port=3312 precaching index 'rt' precached 1 indexes in 0.001 sec [ OK ] # service searchd status searchd (pid 15041) is running... Note that your file doesn’t contain magics=1. So I totally agree that it’s an annoying bug that the developers should fix, but it’s also something you can avoid by putting it in the file you output. Since it starts up fine without the line, and would fail to start on Sphinx 2 or 2.2 with the line, I'm unconvinced on adding it to the default configuration. Administrators who are running Sphinx 2.0.x can simply heed the warning and add compat_sphinxql_magics=0 to their configuration. - Alex
Re: [rt-users] docs improvement suggestion for full-text searching
Sphinx refuses to run without that parameter. Which given that it wasn’t defined in the file they really should have set the default appropriately and not whined at the user, but this is the version of Sphinx currently in RHEL EPEL so there’s going to be a lot of RHEL/CentOS users running into this problem. On Dec 5, 2014, at 8:07 AM, Alex Vandiver ale...@bestpractical.com wrote: On 12/01/2014 04:11 PM, Jo Rhett wrote: version 2.0.8-1 rpm package for EL6 compat_sphinxql_magics was added in 2.0.1-beta, defaults to 0 in 2.1.1-beta, and was removed in 2.2.1-beta. I'm hesitant to add something to the documented configuration which will cause sphinx to fail on all other versions. Can you explain the failure mode of not having it more clearly? binlog_path was added in 1.10 -- I've added it, with a comment to the versions it's pertinent to. - Alex -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On 12/08/2014 01:12 PM, Jo Rhett wrote:
Sphinx refuses to run without that parameter. Which given that it
wasn’t defined in the file they really should have set the default
appropriately and not whined at the user, but this is the version of
Sphinx currently in RHEL EPEL so there’s going to be a lot of
RHEL/CentOS users running into this problem.
I can't replicate the compat_sphinxql_magics problems you report with a
stock Sphinx 2.0.8 from EPEL on CentOS 6. With a stock configuration as
provided by 4.2-trunk, indexer runs with no errors (see below). If it
refuses to run, please show your configuration file, the sphinx version,
and the actual error when running without compat_sphinxql_magics.
- Alex
-bash-4.1# rm /opt/rt4/var/sphinx/*
-bash-4.1# indexer --config tmp.conf rt
Sphinx 2.0.8-id64-release (r3831)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file 'tmp.conf'...
indexing index 'rt'...
WARNING: Attribute count is 0: switching to none docinfo
collected 1 docs, 0.0 MB
sorted 0.0 Mhits, 100.0% done
total 1 docs, 626 bytes
total 0.009 sec, 64073 bytes/sec, 102.35 docs/sec
total 2 reads, 0.000 sec, 0.3 kb/call avg, 0.0 msec/call avg
total 6 writes, 0.000 sec, 0.3 kb/call avg, 0.0 msec/call avg
-bash-4.1# cat tmp.conf
source rt {
type= mysql
sql_host= 127.0.0.1
sql_db = rt4
sql_user= root
sql_pass=
sql_query_pre = SET NAMES utf8
sql_query = \
SELECT a.id, a.content FROM Attachments a \
JOIN Transactions txn ON a.TransactionId = txn.id AND
txn.ObjectType = 'RT::Ticket' \
JOIN Tickets t ON txn.ObjectId = t.id \
WHERE a.ContentType = 'text/plain' AND t.Status != 'deleted'
sql_query_info = SELECT * FROM Attachments WHERE id=$id
}
index rt {
source = rt
path= /opt/rt4/var/sphinx/index
docinfo = extern
charset_type= utf-8
}
indexer {
mem_limit = 32M
}
searchd {
port= 3312
log = /opt/rt4/var/sphinx/searchd.log
query_log = /opt/rt4/var/sphinx/query.log
read_timeout= 5
max_children= 30
pid_file= /opt/rt4/var/sphinx/searchd.pid
max_matches = 1000
seamless_rotate = 1
preopen_indexes = 0
unlink_old = 1
}
Re: [rt-users] docs improvement suggestion for full-text searching
On 12/01/2014 04:11 PM, Jo Rhett wrote: version 2.0.8-1 rpm package for EL6 compat_sphinxql_magics was added in 2.0.1-beta, defaults to 0 in 2.1.1-beta, and was removed in 2.2.1-beta. I'm hesitant to add something to the documented configuration which will cause sphinx to fail on all other versions. Can you explain the failure mode of not having it more clearly? binlog_path was added in 1.10 -- I've added it, with a comment to the versions it's pertinent to. - Alex
Re: [rt-users] docs improvement suggestion for full-text searching
On 11/28/2014 03:12 AM, Jo Rhett wrote: Understood. Just a clarity nitpick :) Although I am confused by your statements that full text indexing isn’t available in MySQL. I’ve used this quite successfully in the past http://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html At the time the documentation was written, MySQL only had FTS support using MyISAM, not InnoDB. Commits 84066c4 and 77641fc on the unmerged 4.2/mysql-native-fts branch clarify the topic, as well as implement native FTS support. Likewise, MariaDB has Sphinx support compiled in and available in their packages, which made this process almost trivial to enable. https://mariadb.com/kb/en/mariadb/documentation/storage-engines/sphinx-storage-engine/about-sphinxse/ RT doesn't officially support MariaDB -- though this is almost purely because of lack of documentation and testing infrastucture, not because it is known to have failure modes. When MariaDB support is added, the Sphinx integration will be noted. I found three problems in the config pushed out by the sbin/rt-setup-fulltext-index command: 1. You need to create and chown the var/sphinx directory it references mkdir /opt/rt4/var/sphinx chown -R sphinx:sphinx /opt/rt4/var/sphinx Noted in 2. You need to add this binlog path line binlog_path = /opt/rt4/var/sphinx 3. You need to add this compat line or searchd won’t start compat_sphinxql_magics = 0 What version of sphinx? Neither of those are necessary with 2.0. - Alex
Re: [rt-users] docs improvement suggestion for full-text searching
version 2.0.8-1 rpm package for EL6 On Dec 1, 2014, at 9:21 AM, Alex Vandiver ale...@bestpractical.com wrote: On 11/28/2014 03:12 AM, Jo Rhett wrote: Understood. Just a clarity nitpick :) Although I am confused by your statements that full text indexing isn’t available in MySQL. I’ve used this quite successfully in the past http://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html At the time the documentation was written, MySQL only had FTS support using MyISAM, not InnoDB. Commits 84066c4 and 77641fc on the unmerged 4.2/mysql-native-fts branch clarify the topic, as well as implement native FTS support. Likewise, MariaDB has Sphinx support compiled in and available in their packages, which made this process almost trivial to enable. https://mariadb.com/kb/en/mariadb/documentation/storage-engines/sphinx-storage-engine/about-sphinxse/ RT doesn't officially support MariaDB -- though this is almost purely because of lack of documentation and testing infrastucture, not because it is known to have failure modes. When MariaDB support is added, the Sphinx integration will be noted. I found three problems in the config pushed out by the sbin/rt-setup-fulltext-index command: 1. You need to create and chown the var/sphinx directory it references mkdir /opt/rt4/var/sphinx chown -R sphinx:sphinx /opt/rt4/var/sphinx Noted in 2. You need to add this binlog path line binlog_path = /opt/rt4/var/sphinx 3. You need to add this compat line or searchd won’t start compat_sphinxql_magics = 0 What version of sphinx? Neither of those are necessary with 2.0. - Alex -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On Nov 27, 2014, at 6:55 AM, Alex Vandiver ale...@bestpractical.com wrote: On 11/26/2014 06:35 PM, Jo Rhett wrote: However [doc/full_text_indexing] contains only information on how to enable full text indexing, and no information on how to enable it without the indexing. Mostly because we're afraid of users shooting themselves in the foot. Regardless, I've committed 2551dca5 which details how to do so. - Alex Understood. Just a clarity nitpick :) Although I am confused by your statements that full text indexing isn’t available in MySQL. I’ve used this quite successfully in the past http://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html Likewise, MariaDB has Sphinx support compiled in and available in their packages, which made this process almost trivial to enable. https://mariadb.com/kb/en/mariadb/documentation/storage-engines/sphinx-storage-engine/about-sphinxse/ I found three problems in the config pushed out by the sbin/rt-setup-fulltext-index command: 1. You need to create and chown the var/sphinx directory it references mkdir /opt/rt4/var/sphinx chown -R sphinx:sphinx /opt/rt4/var/sphinx 2. You need to add this binlog path line binlog_path = /opt/rt4/var/sphinx 3. You need to add this compat line or searchd won’t start compat_sphinxql_magics = 0 Easy fixes… -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
Re: [rt-users] docs improvement suggestion for full-text searching
On 11/26/2014 06:35 PM, Jo Rhett wrote: However [doc/full_text_indexing] contains only information on how to enable full text indexing, and no information on how to enable it without the indexing. Mostly because we're afraid of users shooting themselves in the foot. Regardless, I've committed 2551dca5 which details how to do so. - Alex
[rt-users] docs improvement suggestion for full-text searching
So the documentation at https://www.bestpractical.com/docs/rt/4.2/UPGRADING-4.0.html says: Since 4.0.0, RT's ticket content search is disabled by default because of performance issues when used without full text indexing. For details on how to re-enable it with (or without) full text indexing, see docs/full_text_indexing.pod. However this URL https://www.bestpractical.com/docs/rt/4.2/full_text_indexing.html contains only information on how to enable full text indexing, and no information on how to enable it without the indexing. Seems like you need to add the information from http://blog.bestpractical.com/2011/06/full-text-searching.html to the top of this page. -- Jo Rhett +1 (415) 999-1798 Skype: jorhett Net Consonance : net philanthropy to improve open source and internet projects.
