Re: FTS config - cannot search substrings

2021-11-18 Thread Shawn Heisey

On 11/18/2021 4:43 PM, infoomatic wrote:

sometimes ignoring the problem for a few moments and then starting over
again helps ...

the solution was already solved here in a mail over 10 years ago -
https://dovecot.org/list/dovecot/2011-May/059338.html

Thanks Daniel Miller!


There's something to keep in mind here.  This is the critical piece that 
makes the substring search work:




Because the minimum size is 3, that config means that it is impossible 
to search for one or two letter words -- terms shorter than minGramSize 
are dropped.  This could affect the quality of search results.


If you are running at least version 7.4.0 of Solr, you can add this to 
the config for that filter so that isn't a problem:


preserveOriginal="true"

https://javadoc.io/static/org.apache.lucene/lucene-analyzers-common/7.4.0/org/apache/lucene/analysis/ngram/NGramFilterFactory.html

Thanks,
Shawn


Re: FTS config - cannot search substrings

2021-11-18 Thread infoomatic

sometimes ignoring the problem for a few moments and then starting over
again helps ...

the solution was already solved here in a mail over 10 years ago -
https://dovecot.org/list/dovecot/2011-May/059338.html

Thanks Daniel Miller!


On 18.11.21 23:58, infoomatic wrote:

Hi,

I have an issue with FTS, maybe someone here can help me out.

Basically my setup is working with solr + tika enabled, however, it
seems my search is not configured the way I expect it. Eg when I receive
a pdf document containing the word "myfoobar" I cannot search for
"myfoo" or "foobar", but searching for "myfoobar" shows the right result.

Any ideas? Thanks!




FTS config - cannot search substrings

2021-11-18 Thread infoomatic

Hi,

I have an issue with FTS, maybe someone here can help me out.

Basically my setup is working with solr + tika enabled, however, it
seems my search is not configured the way I expect it. Eg when I receive
a pdf document containing the word "myfoobar" I cannot search for
"myfoo" or "foobar", but searching for "myfoobar" shows the right result.

Any ideas? Thanks!




Re: Solr FTS - message deletes not working as expected

2021-11-18 Thread Shawn Heisey

On 11/3/21 11:45 PM, Shawn Heisey wrote:
Manual expunges of existing messages also are not sending a delete 
request to Solr.  I waited several minutes for that too. 


Update on this, since you're all on the edge of your seat waiting. :)

I did a Send test with TypeApp, a mail client for Android.  It behaved 
completely as expected:  Dovecot immediately sent a delete request to 
Solr for the temporary message in Drafts.


So I went to Mozilla forums and started a discussion about what I am 
seeing with Thunderbird.


I did another test run with sending a message with unique text, and 
doing a manual Solr query for that unique text, I saw one result before 
sending and two results after sending.  Watching the Solr log, no delete 
request was sent.


Then something completely unexpected:  I needed to reboot the laptop, so 
I quit Thunderbird, and I hadn't yet closed my ssh session to the mail 
server, which was still tailing the solr log with a grep for delete 
messages.  As soon as I did the quit, Solr got a delete request for the 
Drafts message.  I did another test -- shift-delete a message, see that 
Solr did not see the delete request.  And then after I waited a while, I 
quit Thunderbird again.  Instantly got a delete request in Solr's log.


Definitely a Thunderbird problem, not dovecot.  Thanks for your 
patience, everyone.  I will pursue it with Mozilla now.


Thanks,
Shawn




Re: Can sieve-script run when saving mail in Sent folder?

2021-11-18 Thread Kees van Vloten

On 18-11-2021 01:01, Gedalya wrote:


On 11/18/21 06:43, Kees van Vloten wrote:

I was expecting through imapsieve_mailboxXXX_* but can find a 
combination of parameters are correct. Does anybody have this 
working? and how? 



plugin {

  sieve_plugins = sieve_imapsieve

  imapsieve_mailbox1_name = Sent
  imapsieve_mailbox1_causes = APPEND COPY
  imapsieve_mailbox1_before = file:~/sieve/IMAP-Sent.sieve

}


protocol imap {
  mail_plugins = $mail_plugins imap_sieve
}

This works fine. What exactly is your issue?





I use Thunderbird 78.14.0 (on Debian Bullseye) as the mail-client to 
test with. After sending a message the copy is stored in Sent but not 
moved to Sent/example.com.


There is no trace in the logs that it did try to run the sieve-script 
(can I enable specific debug logging for this?)



I ran the script manually with sieve-filter (after removing requirements 
"copy" and "imapsieve") it worked properly and moved the messages.



vmail@server:~$ ls -l /var/lib/dovecot/sieve/imap/sent_mail_per_domain.sieve
-rw--- 1 vmail vmail 140 Nov 17 21:00 
/var/lib/dovecot/sieve/imap/sent_mail_per_domain.sieve



vmail@server:~$ cat /var/lib/dovecot/sieve/imap/sent_mail_per_domain.sieve
require ["fileinto", "variables", "mailbox", "copy", "imapsieve"];

if header :matches "From" "*@*>?" {
    fileinto :create "Sent~${2}";
}

Note that tilde (~) is the defined separator in "namespace inbox".


dovecot -n
# 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.13 (cdd19fe3)
# OS: Linux 5.10.0-9-amd64 x86_64 Debian 11.1
# Hostname: server.example.com
auth_default_realm = EXAMPLE.COM
auth_gssapi_hostname = server.example.com
auth_krb5_keytab = /etc/keytab/dovecot.keytab
auth_master_user_separator = *
auth_mechanisms = gssapi gss-spnego plain
auth_realms = EXAMPLE.COM
disable_plaintext_auth = no
first_valid_gid = 986
first_valid_uid = 990
imap_client_workarounds = tb-extra-mailbox-sep
login_greeting = Dovecot ready.
mail_debug = yes
mail_gid = 986
mail_location = 
maildir:%h/%d/%n/store:LAYOUT=fs:FULLDIRNAME=0_FolderContent:UTF-8:INDEX=%h/%d/%n/index:CONTROL=%h/%d/%n/control:VOLATILEDIR=%h/%d/%n/volatile

mail_plugins = zlib notify push_notification listescape acl fts fts_xapian
mail_shared_explicit_inbox = yes
mail_uid = 990
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext vacation-seconds spamtest imapflags notify imapsieve 
vnd.dovecot.imapsieve

namespace inbox {
  inbox = yes
  list = yes
  location =
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Inbox {
    auto = subscribe
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix =
  separator = ~
  subscriptions = yes
  type = private
}
passdb {
  args = username_format=%u /etc/dovecot/master-users
  driver = passwd-file
  master = yes
}
passdb {
  args = /etc/dovecot/ldap_user_to_principal.conf.ext
  driver = ldap
  pass = yes
}
passdb {
  driver = pam
}
plugin {
  acl = vfile:/var/lib/dovecot/global-acls:cache_secs=1
  acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes
  autocreate = Trash
  autocreate2 = Drafts
  autocreate3 = Sent
  autosubscribe = Trash
  autosubscribe2 = Drafts
  autosubscribe3 = Sent
  fts = xapian
  fts_autoindex = yes
  fts_autoindex_exclude = \Trash
  fts_decoder = decode2text
  fts_enforced = yes
  fts_xapian = partial=3 full=20
  imapsieve_mailbox1_before = 
file:/var/lib/dovecot/sieve/imap/learn-spam.sieve

  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox2_before = 
file:/var/lib/dovecot/sieve/imap/learn-ham.sieve

  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_name = *
  imapsieve_mailbox3_before = 
file:/var/lib/dovecot/sieve/imap/sent_mail_per_domain.sieve

  imapsieve_mailbox3_causes = APPEND COPY
  imapsieve_mailbox3_name = Sent
  listescape_char = \
  sieve = file:%h/%d/%n/sieve/sieve;active=%h/%d/%n/sieve/active.sieve
  sieve_after = /var/lib/dovecot/sieve/after
  sieve_before = /var/lib/dovecot/sieve/before
  sieve_default = /var/lib/dovecot/sieve/default.sieve
  sieve_extensions = +vacation-seconds +reject +notify +imapflags +spamtest
  sieve_global = /var/lib/dovecot/sieve/global_include
  sieve_global_extensions = +vnd.dovecot.pipe
  sieve_pipe_bin_dir = /var/lib/dovecot/sieve/bin
  sieve_plugins = sieve_imapsieve sieve_extprograms
  sieve_vacation_default_period = 1d
  sieve_vacation_max_period = 1d
  sieve_vacation_min_period = 10s
  sieve_vacation_use_original_recipient = yes
}
postmaster_address = sysad...@example.com
protocols = " imap lmtp sieve submission"