Re: Cronjobs - change indexing interval

[BKH - Post]

Hello Janos,

everything went fine so far and all email are shown in WebGUI again.
Since i'm in a testing environement it worked for me.
I'm going to test this on a productiv system in the next days.

Thanks for this great piece of software an yout help.
I really appreciate this.

BKH

Am 25.08.19 um 19:17 schrieb s...@acts.hu:
>
>
> Hello,
>
> On 2019-08-25 18:29, BKH wrote:
>>
>> unfortunately all older emails were gone in WebGUI.
>> After checking sphinx index i found them in spinx index.
>> I dont know how to get then back to WebGUI an so i tried to reindex.
>> But this did not work for me.
>>
>> In fact that this is not a big problem - yes, i made some backups - i'm
>> going to set up a new testing environment.
>> But how to get back emails when they are gone in WebGUI but are found in
>> sphinx index?
>>
>
> it appears that the index data defined for sphinx v3 is not compatible
> with
> v2 (and vice versa). So do the following:
>
> - stop searchd
> - rm -f /var/piler/sphinx/*
> - su -c "indexer --all --config /usr/local/etc/piler/sphinx.conf" piler
> - verify that you have files owned by piler in /var/piler/sphinx/
> - startd searchd
> - reindex all emails, ie. su - piler; cd /tmp; reindex -f 1 -t 100;
> /usr/local/libexec/piler/indexer.delta.sh
>
> After this you should see some emails in the gui. If so, run reindex
> -a (as user piler)
> to reindex all emails.
>
> Janos
>

Re: Cronjobs - change indexing interval

[sj]

Hello,

On 2019-08-25 18:29, BKH wrote:


unfortunately all older emails were gone in WebGUI.
After checking sphinx index i found them in spinx index.
I dont know how to get then back to WebGUI an so i tried to reindex.
But this did not work for me.

In fact that this is not a big problem - yes, i made some backups - i'm
going to set up a new testing environment.
But how to get back emails when they are gone in WebGUI but are found 
in

sphinx index?



it appears that the index data defined for sphinx v3 is not compatible 
with

v2 (and vice versa). So do the following:

- stop searchd
- rm -f /var/piler/sphinx/*
- su -c "indexer --all --config /usr/local/etc/piler/sphinx.conf" piler
- verify that you have files owned by piler in /var/piler/sphinx/
- startd searchd
- reindex all emails, ie. su - piler; cd /tmp; reindex -f 1 -t 100; 
/usr/local/libexec/piler/indexer.delta.sh


After this you should see some emails in the gui. If so, run reindex -a 
(as user piler)

to reindex all emails.

Janos

Re: Cronjobs - change indexing interval

[BKH]

Ok, it worked, thank again :-)

Is this spinx.conf part of piler download (Version 1.3.5) which i used?

I changed @LOCALSTATEDIR@ to /var

Then i tried to delete older mails / deleted mails and it worked.
But i got a FAIL notice:

FATAL: failed to merge index 'delta1' into index 'dailydelta1': kbatch
target 'main1' from merge source index must also be in destination index

But all marked mail are gone and not shown in search list.

Is this a severe fault or can it be ignored?


Am 24.08.19 um 16:31 schrieb s...@acts.hu:
>
>
> Hello,
>
> On 2019-08-24 16:11, BKH wrote:
>>
>> i put
>>
>> define('SPHINX_VERSION', 311);
>
> if a similar line wasn't there already, then your sphinx.conf version is
> not ready for 3.1.1. Grab this one:
> https://bitbucket.org/jsuto/piler/raw/c84bea95061820135cc1edcc8beb88ccb0d0f053/etc/sphinx.conf.in
>
> and save it as sphinx.conf (note that @LOCALSTATEDIR@ must be fixed at
> the beginning).
>
> Set 311 for the sphinx version, then check if searchd is happy with it:
> (it will print any issues it encounters)
>
> searchd -c /usr/local/etc/piler/sphinx.conf
>
> if so, then kill it, and restart it with the rc script or systemctl.
>
> Janos
>

Re: Cronjobs - change indexing interval

[BKH]

Thanks, i give i a try

LOCALSTAEDIR defautls to sphinxDIR?


Am 24.08.19 um 16:31 schrieb s...@acts.hu:
>
>
> Hello,
>
> On 2019-08-24 16:11, BKH wrote:
>>
>> i put
>>
>> define('SPHINX_VERSION', 311);
>
> if a similar line wasn't there already, then your sphinx.conf version is
> not ready for 3.1.1. Grab this one:
> https://bitbucket.org/jsuto/piler/raw/c84bea95061820135cc1edcc8beb88ccb0d0f053/etc/sphinx.conf.in
>
> and save it as sphinx.conf (note that @LOCALSTATEDIR@ must be fixed at
> the beginning).
>
> Set 311 for the sphinx version, then check if searchd is happy with it:
> (it will print any issues it encounters)
>
> searchd -c /usr/local/etc/piler/sphinx.conf
>
> if so, then kill it, and restart it with the rc script or systemctl.
>
> Janos
>

Re: Cronjobs - change indexing interval

[sj]

Hello,

On 2019-08-24 16:11, BKH wrote:


i put

define('SPHINX_VERSION', 311);


if a similar line wasn't there already, then your sphinx.conf version is
not ready for 3.1.1. Grab this one:
https://bitbucket.org/jsuto/piler/raw/c84bea95061820135cc1edcc8beb88ccb0d0f053/etc/sphinx.conf.in
and save it as sphinx.conf (note that @LOCALSTATEDIR@ must be fixed at 
the beginning).


Set 311 for the sphinx version, then check if searchd is happy with it:
(it will print any issues it encounters)

searchd -c /usr/local/etc/piler/sphinx.conf

if so, then kill it, and restart it with the rc script or systemctl.

Janos

Re: Cronjobs - change indexing interval

[BKH]

Hello Janos,

this is a great idea and it did the job.
I was kind of confused by changing cronjobs seeing no changes in WebGUI.
Thanks a lot!

BKH


>
>
> Hello,
>
> On 2019-08-24 12:55, BKH wrote:
>>
>> thanks again for your advice.
>>
>> I changed values according to your suggestion and it works.
>>
>> But i stumbled upon WebGUI, where indexer job is shown as a regular
>> 30-minute-job, even if values are changed in piler cronjobs.
>> Is there some workaround for this?
>
> well, it's somewhat misleading as it's the 'indexer job' line is
> static in a sense that it simply adds 30 mins to the last modified
> date of the delta indexer file.
>
> To fix it edit model/health/health.php, find function indexer_stat()
> at the end, and fix +30*60 to +10*60. Perhaps this should be a variable
> in config.php as well.
>
> Janos
>

Re: Cronjobs - change indexing interval

[BKH]

Hi Janos,

i put

define('SPHINX_VERSION', 311);

below

#
# minimal sphinx configuration suited to piler
#

and over

source base
{
   type = mysql
   sql_host = localhost
   sql_db = piler
   sql_user = piler
   sql_pass = 0m8k1inn.

   sql_attr_uint = size
   sql_attr_uint = sent
   sql_attr_uint = attachments
}


after

systemctl restart rc.searchd.service i get:

systemctl status rc.searchd.service
● rc.searchd.service - LSB: sphinxsearch
   Loaded: loaded (/etc/init.d/rc.searchd; generated)
   Active: failed (Result: exit-code) since Sat 2019-08-24 16:08:49
CEST; 7s ago
 Docs: man:systemd-sysv-generator(8)
  Process: 7901 ExecStart=/etc/init.d/rc.searchd start (code=exited,
status=1/FAILURE)

Aug 24 16:08:49 piler01 rc.searchd[7901]: [Sat Aug 24 16:08:49.752 2019]
[7906] FATAL: failed to parse config file '/usr/local/etc/piler/sphin
Aug 24 16:08:49 piler01 rc.searchd[7901]: [Sat Aug 24 16:08:49.752 2019]
[7906] shutdown complete
Aug 24 16:08:49 piler01 rc.searchd[7901]: Sphinx 3.1.1 (commit 612d99f)
Aug 24 16:08:49 piler01 rc.searchd[7901]: Copyright (c) 2001-2018,
Andrew Aksyonoff
Aug 24 16:08:49 piler01 rc.searchd[7901]: Copyright (c) 2008-2016,
Sphinx Technologies Inc (http://sphinxsearch.com)
Aug 24 16:08:49 piler01 rc.searchd[7901]: ERROR: invalid section type
'define' in /usr/local/etc/piler/sphinx.conf line 5 col 7.
Aug 24 16:08:49 piler01 su[7904]: pam_unix(su:session): session closed
for user piler
Aug 24 16:08:49 piler01 systemd[1]: rc.searchd.service: Control process
exited, code=exited, status=1/FAILURE
Aug 24 16:08:49 piler01 systemd[1]: rc.searchd.service: Failed with
result 'exit-code'.
Aug 24 16:08:49 piler01 systemd[1]: Failed to start LSB: sphinxsearch.

What to do now?

Am 24.08.19 um 15:46 schrieb s...@acts.hu:
>
>
> Hello,
>
> On 2019-08-24 12:55, BKH wrote:
>>
>> thanks again for your advice.
>>
>> I changed values according to your suggestion and it works.
>>
>> But i stumbled upon WebGUI, where indexer job is shown as a regular
>> 30-minute-job, even if values are changed in piler cronjobs.
>> Is there some workaround for this?
>
> well, it's somewhat misleading as it's the 'indexer job' line is
> static in a sense that it simply adds 30 mins to the last modified
> date of the delta indexer file.
>
> To fix it edit model/health/health.php, find function indexer_stat()
> at the end, and fix +30*60 to +10*60. Perhaps this should be a variable
> in config.php as well.
>
> Janos
>

Re: Cronjobs - change indexing interval

[BKH]

Hello Janos,

thanks again for your advice.

I changed values according to your suggestion and it works.

But i stumbled upon WebGUI, where indexer job is shown as a regular
30-minute-job, even if values are changed in piler cronjobs.
Is there some workaround for this?

BKH

Am 24.08.19 um 12:19 schrieb Janos SUTO:
> Hello,
>
> try this:
>
> */10 * * * * 
>
> Btw. check the mail log to see if it works as required. Whenever the
> indexer script starts it syslogs it.
>
> Janos
> 
> *From:* BKH
> *Sent:* Fri Aug 23 21:35:04 GMT+02:00 2019
> *To:* piler-user@list.acts.hu
> *Subject:* Cronjobs - change indexing interval
>
>
> Hello,
>
> for testing reasons i tried changing indexing interval.
> I played around with this line:
>
> 5,35 * * * * /usr/local/libexec/piler/indexer.delta.sh 
> 
>
> I changed minute an hour to differnte values, but could not figure out
> to get it in a working state.
> Indexing should take place e.g. every 10 minutes.
> How can i achive this?
>
> Thanks :-)
>
> BKH
>
>
>

Re: Cronjobs - change indexing interval

[Janos SUTO]

Hello,

try this:

*/10 * * * * 

Btw. check the mail log to see if it works as required. Whenever the indexer 
script starts it syslogs it.

Janos


 Original Message 
From: BKH 
Sent: Fri Aug 23 21:35:04 GMT+02:00 2019
To: piler-user@list.acts.hu
Subject: Cronjobs - change indexing interval



Hello,

for testing reasons i tried changing indexing interval.
I played around with this line:

5,35 * * * * /usr/local/libexec/piler/indexer.delta.sh

I changed minute an hour to differnte values, but could not figure out
to get it in a working state.
Indexing should take place e.g. every 10 minutes.
How can i achive this?

Thanks :-)

BKH