Yes, please. Reindex works just as before. Janos
On Mar 19, 2023, 20:47, at 20:47, Jonathan Baecker <[email protected]> wrote: > > >Thank you, that was the trick! But I still have to run: reindex -a? > >Am 19.03.23 um 20:44 schrieb Janos SUTO: >> >> >> From your previous emails, you set real-time (rt) indexing. For such >> setup, >> don't run the indexer utility, no need for it any longer. >> >> Also be sure to update the pilersearch.service as in >> >https://bitbucket.org/jsuto/piler/src/master/systemd/pilersearch.service >> >> I think you'll be fine after that, manticore will create the required >> index files. >> >> Janos >> >> On 2023-03-19 20:36, Jonathan Baecker wrote: >>> This folder is empty and I also can not reindex. I recover my system >>> to a state where it was works. Maybe I miss a step, so here are my >>> steps: >>> >>> 1. piler 1.4.3 runs with sphinx under debian 11. >>> 2. I install manticore 6.0.x from webpage >>> 3. cp /usr/local/etc/piler/manticore.conf.dist >>> /usr/local/etc/piler/manticore.conf >>> 4. add db settings to /usr/local/etc/piler/manticore.conf >>> 4. apply all changes from mailpiler doc about switching to manticore >>> 5. when I run as piler user: indexer --all --config >>> /usr/local/etc/piler/manticore.conf it gives me: FATAL: no tables >>> found in config file '/usr/local/etc/piler/manticore.conf' >>> 6. I reboot the system to have all services running >>> >>> systemctl status pilersearch.service show: >>> >>> Process: 611 ExecStart=/usr/bin/searchd --config >>> /usr/local/etc/piler/sphinx.conf (code=exited, status=0/SUCCESS) >>> Main PID: 615 (searchd) >>> Tasks: 10 (limit: 4675) >>> Memory: 52.2M >>> CPU: 314ms >>> CGroup: /system.slice/pilersearch.service >>> ├─614 /usr/bin/searchd --config >>> /usr/local/etc/piler/sphinx.conf >>> ├─615 /usr/bin/searchd --config >>> /usr/local/etc/piler/sphinx.conf >>> └─620 /usr/bin/manticore-executor -n >>> /tmp/manticore-buddy.phar --listen=http://127.0.0.1:9312 --threads=2 >>> >>> Mär 19 20:27:40 piler searchd[611]: Copyright (c) 2008-2016, Sphinx >>> Technologies Inc (http://sphinxsearch.com) >>> Mär 19 20:27:40 piler searchd[611]: Copyright (c) 2017-2023, >Manticore >>> Software LTD (https://manticoresearch.com) >>> Mär 19 20:27:40 piler searchd[611]: WARNING: key 'docinfo' was >>> permanently removed from configuration. Refer to documentation for >>> details. >>> >>> So it still uses the old sphinx.conf >>> >>> Was my steps ok? >>> >>> >>> >>> Am 19.03.23 um 18:48 schrieb Janos SUTO: >>>> >>>> >>>> Your configs look fine. The errno=1064 means that no such local >index. >>>> Show me "ls -la /var/piler/manticore/" output, too. >>>> >>>> In my test environment it looks like: >>>> >>>> ls -la /var/piler/manticore/ >>>> total 4796 >>>> drwx------ 2 piler piler 4096 Mar 19 17:43 . >>>> drwxr-xr-x 10 root root 4096 Mar 19 17:01 .. >>>> -rw------- 1 piler piler 8 Mar 19 17:43 binlog.001 >>>> -rw------- 1 piler piler 0 Mar 19 17:03 binlog.lock >>>> -rw------- 1 piler piler 11 Mar 19 17:43 binlog.meta >>>> -rw------- 1 piler piler 2731 Mar 19 17:43 manticore.log >>>> -rw------- 1 piler piler 0 Mar 19 17:03 note1.lock >>>> -rw------- 1 piler piler 1200 Mar 19 17:03 note1.meta >>>> -rw------- 1 piler piler 20 Mar 19 17:03 note1.ram >>>> -rw------- 1 piler piler 0 Mar 19 17:03 piler1.lock >>>> -rw------- 1 piler piler 1569 Mar 19 17:43 piler1.meta >>>> -rw------- 1 piler piler 4862221 Mar 19 17:43 piler1.ram >>>> -rw------- 1 piler piler 893 Mar 19 17:01 query.log >>>> -rw------- 1 piler piler 0 Mar 19 17:03 tag1.lock >>>> -rw------- 1 piler piler 1199 Mar 19 17:03 tag1.meta >>>> -rw------- 1 piler piler 20 Mar 19 17:03 tag1.ram >>>> >>>> Janos >>>> >>>> On 2023-03-19 17:15, Jonathan Baecker wrote: >>>>> Ah sorry! Here is the output, but I run: >>>>> /usr/local/etc/piler/manticore.conf >>>>> >>>>> # >>>>> # minimal manticore configuration suited to piler >>>>> # >>>>> >>>>> >>>>> index piler1 >>>>> { >>>>> type = rt >>>>> path = /var/piler/manticore/piler1 >>>>> rt_mem_limit = 512M >>>>> stored_fields = >>>>> min_word_len = 1 >>>>> min_prefix_len = 5 >>>>> charset_table = 0..9, english, _, \ >>>>> U+C1->U+E1, U+C4->U+E4, >U+C5->U+E5, >>>>> U+C6->U+E6, U+C9->U+E9, U+CD->U+ED, U+D3->U+F3, U+D6->U+F6, >>>>> U+D8->U+F8, \ >>>>> U+DA->U+FA, U+DC->U+FC, >>>>> U+0150->U+0151, U+0152->U+0153, U+0170->U+0171, U+01E2->U+E6, >>>>> U+01E3->U+E6, U+01FC->U+E6, \ >>>>> U+01FD->U+E6, U+1D01->U+E6, >>>>> U+1D02->U+E6, U+1D2D->U+E6, U+1D46->U+E6, \ >>>>> U+DF, U+E1, U+E4, U+E5, U+E6, >U+E9, >>>>> U+ED, U+00F3, U+F6, U+F8, U+FA, U+FC, U+0151, U+0153, U+0171 >>>>> # See >>>>> >https://manual.manticoresearch.com/Creating_an_index/Data_types#Row-wise-and-columnar-attribute-storages > >>>>> # if you want to enable columnar storage >>>>> # columnar_attrs = * >>>>> rt_field = sender >>>>> rt_field = rcpt >>>>> rt_field = senderdomain >>>>> rt_field = rcptdomain >>>>> rt_field = subject >>>>> rt_field = body >>>>> rt_field = attachment_types >>>>> rt_attr_bigint = arrived >>>>> rt_attr_bigint = sent >>>>> rt_attr_uint = size >>>>> rt_attr_uint = direction >>>>> rt_attr_uint = folder >>>>> rt_attr_uint = attachments >>>>> } >>>>> >>>>> index tag1 >>>>> { >>>>> type = rt >>>>> path = /var/piler/manticore/tag1 >>>>> rt_mem_limit = 16M >>>>> stored_fields = tag >>>>> min_word_len = 2 >>>>> min_prefix_len = 5 >>>>> charset_table = 0..9, english, _, \ >>>>> U+C1->U+E1, U+C4->U+E4, >U+C5->U+E5, >>>>> U+C6->U+E6, U+C9->U+E9, U+CD->U+ED, U+D3->U+F3, U+D6->U+F6, >>>>> U+D8->U+F8, \ >>>>> U+DA->U+FA, U+DC->U+FC, >>>>> U+0150->U+0151, U+0152->U+0153, U+0170->U+0171, U+01E2->U+E6, >>>>> U+01E3->U+E6, U+01FC->U+E6, \ >>>>> U+01FD->U+E6, U+1D01->U+E6, >>>>> U+1D02->U+E6, U+1D2D->U+E6, U+1D46->U+E6, \ >>>>> U+DF, U+E1, U+E4, U+E5, U+E6, >U+E9, >>>>> U+ED, U+00F3, U+F6, U+F8, U+FA, U+FC, U+0151, U+0153, U+0171 >>>>> rt_field = tag >>>>> rt_attr_bigint = mid >>>>> rt_attr_uint = uid >>>>> } >>>>> >>>>> index note1 >>>>> { >>>>> type = rt >>>>> path = /var/piler/manticore/note1 >>>>> rt_mem_limit = 16M >>>>> stored_fields = note >>>>> min_word_len = 2 >>>>> min_prefix_len = 5 >>>>> charset_table = 0..9, english, _, \ >>>>> U+C1->U+E1, U+C4->U+E4, >U+C5->U+E5, >>>>> U+C6->U+E6, U+C9->U+E9, U+CD->U+ED, U+D3->U+F3, U+D6->U+F6, >>>>> U+D8->U+F8, \ >>>>> U+DA->U+FA, U+DC->U+FC, >>>>> U+0150->U+0151, U+0152->U+0153, U+0170->U+0171, U+01E2->U+E6, >>>>> U+01E3->U+E6, U+01FC->U+E6, \ >>>>> U+01FD->U+E6, U+1D01->U+E6, >>>>> U+1D02->U+E6, U+1D2D->U+E6, U+1D46->U+E6, \ >>>>> U+DF, U+E1, U+E4, U+E5, U+E6, >U+E9, >>>>> U+ED, U+00F3, U+F6, U+F8, U+FA, U+FC, U+0151, U+0153, U+0171 >>>>> rt_field = note >>>>> rt_attr_bigint = mid >>>>> rt_attr_uint = uid >>>>> } >>>>> >>>>> searchd >>>>> { >>>>> listen = 127.0.0.1:9312 >>>>> listen = 127.0.0.1:9306:mysql41 >>>>> log = >/var/piler/manticore/manticore.log >>>>> binlog_max_log_size = 256M >>>>> binlog_path = /var/piler/manticore >>>>> binlog_flush = 2 >>>>> query_log = /var/piler/manticore/query.log >>>>> network_timeout = 5 >>>>> pid_file = /var/run/piler/searchd.pid >>>>> seamless_rotate = 1 >>>>> preopen_indexes = 1 >>>>> unlink_old = 1 >>>>> thread_stack = 512k >>>>> rt_flush_period = 300 >>>>> >>>>> } >>>>> >>>>> >>>>> Am 19.03.23 um 16:35 schrieb Janos SUTO: >>>>>> >>>>>> >>>>>> I meant to execute /etc/piler/manticore.conf (yes, it's >executable). >>>>>> >>>>>> Janos >>>>>> >>>>>> On 2023-03-19 16:06, Jonathan Baecker wrote: >>>>>>> Am 19.03.23 um 14:27 schrieb Janos SUTO: >>>>>>>> Hello Jonathan, >>>>>>>> >>>>>>> Hello Jano, thank you for you quick response! >>>>>>> >>>>>>>> please tell me more about your setup. Show me the output of >>>>>>>> >>>>>>>> pilerconf |grep -E '(sph|rtindex)' >>>>>>> >>>>>>> Output is: >>>>>>> >>>>>>> rtindex=1 >>>>>>> sphxdb=piler1 >>>>>>> sphxhost=127.0.0.1 >>>>>>> sphxport=9306 >>>>>>> >>>>>>>> >>>>>>>> and >>>>>>>> >>>>>>>> /etc/piler/manticore.conf >>>>>>> >>>>>>> Content is: >>>>>>> >>>>>>> define('RT', 1); >>>>>>> >>>>>>>> >>>>>>>> Be sure to redact any passwords. >>>>>>>> >>>>>>>> Janos >>>>>>>> >>>>>>>> >>>>>>>> On 2023-03-19 14:11, Jonathan Baecker wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I just update piler from 1.3.9 to 1.4.3 and I also switched to >>>>>>>>> manticore. But now piler can not receive new mails. when I >>>>>>>>> check piler >>>>>>>>> service, I get this error: >>>>>>>>> >>>>>>>>> >>>>>>>>> Mär 19 14:06:52 piler piler[567]: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: hdr_len: 3130, offset: 3 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: touch >>>>>>>>> >2e5fa096927b9a6ac6fe82ff2b81256acfa719eb32031ca68f0d86db45415d9f >>>>>>>>> OK >>>>>>>>> (<[email protected]>) >>>>>>>>> Mär 19 14:06:52 piler piler[567]: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: stored >>>>>>>>> '50000000641708f61c491c9c0069477d77cb.m' 39327/11168 bytes >>>>>>>>> Mär 19 14:06:52 piler piler[567]: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: updated meta reference >for >>>>>>>>> '<FxXfleYASv2VKckQYZ7mWg@geopod-ismtpd-36>', rc=0 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: stored 1 recipients, >rc=0 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: ERROR: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb failed to store index >data >>>>>>>>> for >>>>>>>>> id=31270, errno=1064, append ret=0 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: ERROR: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: rollback sql stmt=DELETE > >>>>>>>>> FROM >>>>>>>>> sph_index WHERE id=31270 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: ERROR: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: rollback sql stmt=DELETE > >>>>>>>>> FROM >>>>>>>>> rcpt WHERE id=31270 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: ERROR: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: rollback sql stmt=DELETE > >>>>>>>>> FROM >>>>>>>>> metadata WHERE id=31270 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: ERROR: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: rollback sql stmt=DELETE > >>>>>>>>> FROM >>>>>>>>> attachment WHERE >piler_id='50000000641708f61c491c9c0069477d77cb' >>>>>>>>> Mär 19 14:06:52 piler piler[567]: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: unlinking >>>>>>>>> >/var/piler/store/00/641/77/cb/50000000641708f61c491c9c0069477d77cb.m >>>>>>>>> >>>>>>>>> Mär 19 14:06:52 piler piler[567]: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb: stored metadata, rc=1 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: 1/E07M4E0192DW6TN0: moved to >>>>>>>>> /var/piler/error/E07M4E0192DW6TN0 >>>>>>>>> Mär 19 14:06:52 piler piler[567]: 1/E07M4E0192DW6TN0: >>>>>>>>> 50000000641708f61c491c9c0069477d77cb, size=39327/11168, >>>>>>>>> attachments=0, >>>>>>>>> reference=<FxXfleYASv2VKckQYZ7mWg@geopod-ismtpd-36>, >>>>>>>>> message-id=<[email protected]>, >>>>>>>>> retention=2557, folder=0, delay=0.1183, status=error >>>>>>>>> >>>>>>>>> >>>>>>>>> After Update I run mysql piler < ./util/db-upgrade.sql But I >>>>>>>>> think it >>>>>>>>> had done nothing. >>>>>>>>> >>>>>>>>> Do you have any idea, how to fix this? >>>>>>>>> >>>>>>>>> >>>>>>>>> Have a good day! >>>>>>>>> >>>>>>>>> jb >>>>>>>> >>>>>> >>>> >>
