Hi, i am running the vbox image and imported 275912 Mails.
I checked some Mails on the webui, but to be sure new added are also shown, i add one message with a subject that was not yet available. The health page counter shows me that the mail was succesfully added, but i could not find it. After waiting more than 8 hours, there was no much cpu usage and no indexer tasks are running in #ps aux | grep ^piler so i decide to reindex all mails .. First i cheked there is no other cronjob of sphinx in /etc/cron.d. Then I commented the crontab of pileruser, looked in sph_index table -> it's empty then i run as user piler in /tmp: /usr/local/bin/indexer --quiet delta1 --rotate to fill up sph_index table. the missing mail i can find now in that table After that all i run as piler in /tmp: /usr/local/bin/indexer --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate it shows me the following: Sphinx 2.0.7-release (r3759) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/usr/local/etc/sphinx.conf'... merging index 'delta1' into index 'main1'... read 82.4 of 82.4 MB, 100.0% done merged 8254.0 Kwords and then it fails with the following output to stdout: Sphinx 2.0.7-release (r3759) Copyright (c) 2001-2012, Andrew Aksyonoff Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com) using config file '/usr/local/etc/sphinx.conf'... merging index 'delta1' into index 'main1'... read 82.4 of 82.4 MB, 100.0% done *** Oops, indexer crashed! Please send the following report to developers. Sphinx 2.0.7-release (r3759) -------------- report begins here --------------- Current document: docid=0, hits=0 Current batch: minid=0, maxid=0 Hit pool start: docid=0, hit=0 -------------- backtrace begins here --------------- Program compiled with gcc 4.4.5 Host OS is Linux debian 2.6.32-5-686 #1 SMP Mon Feb 25 01:04:36 UTC 2013 i686 GNU/Linux Stack bottom = 0xffffffffbfe37e24, thread stack size = 0x10000 begin of manual backtrace: Something wrong with thread stack, backtrace may be incorrect (fp=ffffffffbfe34fc8) Stack looks OK, attempting backtrace. 8099ef4 ffffffffb7370871 811a555 815f2ee 80e124c 809c1d8 80a37f3 ffffffffb7317ca6 8099721 Something wrong in frame pointers, backtrace failed (fp=0) begin of system backtrace: begin of system symbols: /usr/local/bin/indexer(_Z12sphBacktraceib+0x27b)[0x8179ceb] /usr/local/bin/indexer(_Z7sigsegvi+0x104)[0x8099ef4] [0xb77bb400] /lib/i686/cmov/libc.so.6(cfree+0x31)[0xb7370871] /usr/local/bin/indexer(_ZN22DiskIndexQwordTraits_cD2Ev+0xb5)[0x811a555] /usr/local/bin/indexer(_ZN13CSphIndex_VLN10MergeWordsI16DiskIndexQword_cILb1ELb0ELb1EES2_EEbPS_P10ISphFilter+0x65e)[0x815f2ee] /usr/local/bin/indexer(_ZN13CSphIndex_VLN5MergeEP9CSphIndexR10CSphVectorI18CSphFilterSettings16CSphVectorPolicyIS3_EEb+0x278c)[0x80e124c] /usr/local/bin/indexer(_Z7DoMergeRK17CSphConfigSectionPKcS1_S3_R10CSphVectorI18CSphFilterSettings16CSphVectorPolicyIS5_EEbb+0x5a8)[0x809c1d8] /usr/local/bin/indexer(main+0xfb3)[0x80a37f3] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7317ca6] /usr/local/bin/indexer[0x8099721] -------------- backtrace ends here --------------- I decide to give the VM more Ram (8GB instead of default 2GB), but the kernel don't accepts more than 3GB. So i have to install the bigmem kernel image with "apt-get install linux-image-2.6-686-bigmem" After a reboot i can see all 8GB of Ram :) I run the indexer command as piler user again, and it comes up to 15879 KWords Sphinx seems to write to the buffer/cache (free -m the buffer/cache line increases, 3GB of Ram are free at time of failure) Sphinx seems to not handle the Kwords before it reads them all ... ??? Any ideas what i can do now to reindex? Should i remove any files in /var/piler/sphinx ? Best regards Christian
