On 19/10/2020 17:00, PGNet Dev wrote: > On 10/19/20 1:18 AM, John Fawcett wrote: >> I would recommend you to redo the tests after correcting the >> configuration. To be doubly sure you can include accented and unique non >> accented text in the same email and search for both. If the non accented >> text is found you know you've searching against the updated index and >> the fact that accented text is not found is not simply because the index >> updates are not visible. > > temp changing, > > fts_solr = url=https://solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > - fts_enforced = body > + fts_enforced = yes > - fts_filters = normalizer-icu snowball stopwords > + fts_filters = lowercase snowball stopwords > > now, > > doveadm fts rescan -u [email protected] > doveadm index -u [email protected] -q '*' > > , no errors. > > on receipt of a test email with > > subject: aausdfrhyétdwgyatrdf aausdfrhyetdwgyatrdf > body: aausdfrhyétdwgyatrdf aausdfrhyetdwgyatrdf > > i see a solr auto-scan, > > 2020-10-19 14:41:50.628 INFO > (searcherExecutor-15-thread-1-processing-x:dovecot) [ x:dovecot] > o.a.s.c.SolrCore [dovecot] Registered new searcher autowarm time: 1 ms > 2020-10-19 14:41:50.634 INFO (qtp1533985074-18) [ x:dovecot] > o.a.s.u.p.LogUpdateProcessorFactory [dovecot] webapp=/solr > path=/update params={}{commit=} 0 155 > 2020-10-19 14:41:51.571 INFO (qtp1533985074-24) [ x:dovecot] > o.a.s.u.p.LogUpdateProcessorFactory [dovecot] webapp=/solr > path=/update > params={}{add=[135785/c92f64f79f0d1ed01e6d5b314f04886c/[email protected] > (1680991596381732864)]} 0 9 > > ==> /var/log/solr/solr_gc.log <== > [2020-10-19T07:41:51.612-0700][146823.946s] GC(343) Pause Young > (Normal) (G1 Evacuation Pause) > [2020-10-19T07:41:51.613-0700][146823.947s] GC(343) Using 2 > workers of 2 for evacuation > [2020-10-19T07:41:51.635-0700][146823.969s] GC(343) Pre Evacuate > Collection Set: 0.3ms > [2020-10-19T07:41:51.636-0700][146823.970s] GC(343) Merge Heap > Roots: 0.2ms > [2020-10-19T07:41:51.636-0700][146823.970s] GC(343) Evacuate > Collection Set: 18.9ms > [2020-10-19T07:41:51.637-0700][146823.971s] GC(343) Post > Evacuate Collection Set: 2.0ms > [2020-10-19T07:41:51.637-0700][146823.971s] GC(343) Other: 1.5ms > [2020-10-19T07:41:51.637-0700][146823.971s] GC(343) Eden regions: > 238->0(244) > [2020-10-19T07:41:51.638-0700][146823.972s] GC(343) Survivor > regions: 4->2(31) > [2020-10-19T07:41:51.638-0700][146823.972s] GC(343) Old regions: > 189->189 > [2020-10-19T07:41:51.638-0700][146823.973s] GC(343) Archive > regions: 2->2 > [2020-10-19T07:41:51.639-0700][146823.973s] GC(343) Humongous > regions: 10->9 > [2020-10-19T07:41:51.639-0700][146823.973s] GC(343) Metaspace: > 61564K(78028K)->61564K(78028K) NonClass: > 55348K(65024K)->55348K(65024K) Class: 6216K(13004K)->6216K(13004K) > [2020-10-19T07:41:51.640-0700][146823.974s] GC(343) Pause Young > (Normal) (G1 Evacuation Pause) 441M->200M(512M) 27.372ms > [2020-10-19T07:41:51.640-0700][146823.974s] GC(343) User=0.01s > Sys=0.01s Real=0.03s > > ==> /var/log/solr/solr.log <== > 2020-10-19 14:41:51.702 INFO > (searcherExecutor-15-thread-1-processing-x:dovecot) [ x:dovecot] > o.a.s.c.SolrCore [dovecot] Registered new searcher autowarm time: 0 ms > 2020-10-19 14:41:51.705 INFO (qtp1533985074-18) [ x:dovecot] > o.a.s.u.p.LogUpdateProcessorFactory [dovecot] webapp=/solr > path=/update params={}{commit=} 0 127 > > > search in TBird > > subject: aausdfrhyetdwgyatrdf => FOUND > body: aausdfrhyétdwgyatrdf => FOUND > > subject: aausdfrhyetdwgyatrdf => FOUND > body: aausdfrhyétdwgyatrdf => (emtpy) > > on header search, I'm _not_ seeing any additional activity in solr.log > > so, either i'm looking in the wrong place, haven't turned on > appropriate logging, or i'm still not searching via solr ... > > separately, > > doveadm fts lookup ... > > still panics; Aki's bug will hopefully deal with that
Depending how solr has been setup you could see the logging in the web server access log. My access log is where I configured it in /var/log/httpd/servername.access_log, yours may be different. For searches I see things like this (one for each folder searched) 2a01:488:67:1000:523:f8eb:0:1 - john [19/Oct/2020:17:16:39 +0200] "GET /solr/dovecot/select?wt=xml&fl=uid,score&rows=3176&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dbody:aausdfrhy%c3%a9tdwgyatrdf&fq=%2Bbox:b1626f0fe8d9145e54100000c54a863a+%2Buser:[email protected] HTTP/1.1" 200 910 "-" "-" For index updates I see things like this: 2a01:488:67:1000:523:f8eb:0:1 - john [19/Oct/2020:17:10:01 +0200] "POST /solr/dovecot/update HTTP/1.1" 200 156 "-" "-" BTW I've noticed that Thunderbird does not always send the search to the server even with the "search on server" flag set, which is why I look at the access_log in solr if I want to be really sure it's going to solr. John
