Re: Configure: "Can't build with MySQL support: libmysqlclient not found"

2024-03-12 Thread James L. Brown via dovecot


 On 13 Mar 2024, at 12:14 AM, Aki Tuomi 
 wrote:

  On 12/03/2024 14:46 EET James L. Brown via dovecot
   wrote:


  On 12 Mar 2024, at 10:06 PM, Odhiambo Washington
   wrote:

  On Tue, Mar 12, 2024 at 4:20 AM James L. Brown via dovecot
  mailto:dovecot@dovecot.org>> wrote:
  Hi all.

  I’m getting errors when trying to configure make for
  version 2.3.21 on macOS
  14.3.1 Sonoma.

  It ends with:

  checking for mysql_config... mysql_config
  checking for mysql_init in -lmysqlclient... no
  configure: error: Can't build with MySQL support:
  libmysqlclient not found

  I’m using:

  env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config
  --libs libsodium

  CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.2.1/include
  LDFLAGS=-L/opt/
  homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl -
  -with-mysql --with-
  sodium

  Looking at config.log I see:

  configure:28222: checking for mysql_config
  configure:28238: found /opt/homebrew/bin/mysql_config
  configure:28250: result: mysql_config
  configure:28285: checking for mysql_init in -lmysqlclient
  configure:28310: gcc -o conftest -std=gnu99 -g -O2 -fstack-
  protector-strong -
  U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-
  prototypes -Wmissing-
  declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -
  Wbad-function-cast -
  Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/opt/
  homebrew/Cellar/
  openssl@3/3.2.1/include -I/opt/homebrew/Cellar/openssl@3/
  3.2.1/include -L/opt/
  homebrew/opt/openssl@3/lib conftest.c -lmysqlclient   -L/
  opt/homebrew/Cellar/
  mysql/8.3.0_1/lib -lmysqlclient -lz  -lzstd  -lssl  -
  lcrypto -lresolv -lz -lm
   &5
  ld: warning: ignoring duplicate libraries: '-lmysqlclient',
  '-lz'
  ld: library 'zstd' not found
  clang: error: linker command failed with exit code 1 (use -
  v to see invocation)

 You are missing zstd library.

 Aki

Thanks Aki.

zstd 1.5.5 is installed with the other Homebriew-installed stuff at /opt/
homebrew/opt/

Not sure why it is not finding zest or libmysqlclient.

I’ve even just added /opt/homebrew/opt to my PATH:

% echo $PATH                         
/opt/homebrew/opt/openssl@3/bin:/opt/homebrew/sbin:/opt/homebrew/bin:/opt/
homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/
usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/
bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/
bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/
usr/appleinternal/bin:/Library/Apple/usr/bin:/opt/homebrew/opt

James.

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: Configure: "Can't build with MySQL support: libmysqlclient not found"

2024-03-12 Thread James L. Brown via dovecot

 On 12 Mar 2024, at 10:06 PM, Odhiambo Washington 
 wrote:

 On Tue, Mar 12, 2024 at 4:20 AM James L. Brown via dovecot
  wrote:
  Hi all.

  I’m getting errors when trying to configure make for
  version 2.3.21 on macOS
  14.3.1 Sonoma.

  It ends with:

  checking for mysql_config... mysql_config
  checking for mysql_init in -lmysqlclient... no
  configure: error: Can't build with MySQL support:
  libmysqlclient not found

  I’m using:

  env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config
  --libs libsodium

  CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.2.1/include
  LDFLAGS=-L/opt/
  homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl -
  -with-mysql --with-
  sodium

  Looking at config.log I see:

  configure:28222: checking for mysql_config
  configure:28238: found /opt/homebrew/bin/mysql_config
  configure:28250: result: mysql_config
  configure:28285: checking for mysql_init in -lmysqlclient
  configure:28310: gcc -o conftest -std=gnu99 -g -O2 -fstack-
  protector-strong -
  U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-
  prototypes -Wmissing-
  declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -
  Wbad-function-cast -
  Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/opt/
  homebrew/Cellar/
  openssl@3/3.2.1/include -I/opt/homebrew/Cellar/openssl@3/
  3.2.1/include -L/opt/
  homebrew/opt/openssl@3/lib conftest.c -lmysqlclient   -L/
  opt/homebrew/Cellar/
  mysql/8.3.0_1/lib -lmysqlclient -lz  -lzstd  -lssl  -
  lcrypto -lresolv -lz -lm
  >&5
  ld: warning: ignoring duplicate libraries: '-lmysqlclient',
  '-lz'
  ld: library 'zstd' not found
  clang: error: linker command failed with exit code 1 (use -
  v to see invocation)
  configure:28310: $? =1


  Odd that it is using the old version mysql.

  Would someone be able to help me get it to find
  libmysqlclient?

  In /opt/homebrew/Cellar/mysql/8.0.32/lib/ I have:

  libmysqlclient.21.dylib
  libmysqlclient.a
  libmysqlclient.dylib

  Thanks,

      James.


 Maybe https://github.com/PyMySQL/mysqlclient/issues/496#issuecomment-
 1614688099
  
 --
 Best regards,
 Odhiambo WASHINGTON,
 Nairobi,KE
 +254 7 3200 0004/+254 7 2274 3223
  In an Internet failure case, the #1 suspect is a constant: DNS.
 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
 [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-
 questions.html]

Thanks Odhiambo, that looked promising.

I used:

export MYSQLCLIENT_LDFLAGS="-L/opt/homebrew/opt/mysql/lib -lmysqlclient -rpath
/opt/homebrew/opt/mysql/lib" 

Before the env and CPPFLAGS lines.

But I still get:

checking for mysql_init in -lmysqlclient... no
configure: error: Can't build with MySQL support: libmysqlclient not found

And config.log still has in the ‘Output variables’ section:

MYSQL_CONFIG='mysql_config'
MYSQL_LIBS='-L/opt/homebrew/Cellar/mysql/8.3.0_1/lib -lmysqlclient -lz  -lzstd
 -lssl  -lcrypto -lresolv'

I’m sure it’s something like what you suggested.

Thanks, James.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Configure: "Can't build with MySQL support: libmysqlclient not found"

2024-03-11 Thread James L. Brown via dovecot
Hi all.

I’m getting errors when trying to configure make for version 2.3.21 on macOS
14.3.1 Sonoma.

It ends with:

checking for mysql_config... mysql_config
checking for mysql_init in -lmysqlclient... no
configure: error: Can't build with MySQL support: libmysqlclient not found

I’m using:

env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config --libs libsodium

CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.2.1/include LDFLAGS=-L/opt/
homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl --with-mysql --with-
sodium

Looking at config.log I see:

configure:28222: checking for mysql_config
configure:28238: found /opt/homebrew/bin/mysql_config
configure:28250: result: mysql_config
configure:28285: checking for mysql_init in -lmysqlclient
configure:28310: gcc -o conftest -std=gnu99 -g -O2 -fstack-protector-strong -
U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing-
declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -
Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/opt/homebrew/Cellar/
openssl@3/3.2.1/include -I/opt/homebrew/Cellar/openssl@3/3.2.1/include -L/opt/
homebrew/opt/openssl@3/lib conftest.c -lmysqlclient   -L/opt/homebrew/Cellar/
mysql/8.3.0_1/lib -lmysqlclient -lz  -lzstd  -lssl  -lcrypto -lresolv -lz -lm
>&5
ld: warning: ignoring duplicate libraries: '-lmysqlclient', '-lz'
ld: library 'zstd' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)
configure:28310: $? =1


Odd that it is using the old version mysql.

Would someone be able to help me get it to find libmysqlclient?

In /opt/homebrew/Cellar/mysql/8.0.32/lib/ I have:

libmysqlclient.21.dylib
libmysqlclient.a
libmysqlclient.dylib

Thanks,

James.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Dbox-storage error when compiling: no member named 'st_atim' in 'struct stat'

2024-03-02 Thread James L. Brown via dovecot
Hi all.

I’m getting errors when trying to compile 2.3.21 on macOS 14.3.1 Sonoma.

It ends with:

3.2.1/include -MT dbox-storage.lo -MD -MP -MF .deps/dbox-storage.Tpo -c dbox-
storage.c  -fno-common -DPIC -o .libs/dbox-storage.o
dbox-storage.c:296:32:error:no member named 'st_atim' in 'struct stat'
                        last_temp_file_scan = stats.st_atim.tv_sec;
                                              ~ ^
dbox-storage.c:297:24:error:no member named 'st_ctim' in 'struct stat'
                        change_time = stats.st_ctim.tv_sec;
                                      ~ ^
2 errors generated.
make[5]: *** [dbox-storage.lo] Error 1
make[4]: *** [all-recursive] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1

Before ‘make’ I ran:

sudo CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.2.1/include LDFLAGS=-L/opt/
homebrew/opt/openssl@3/lib LDFLAGS=-L/opt/homebrew/lib ./configure --with-
ssl=openssl --with-mysql --with-sodium

I had to run ’sudo’ before ‘make’ as I was getting this error:

error:error opening '.deps/fuzzer.Tpo': /Users/jlbrown/Downloads/dovecot-
2.3.21/src/lib-test/.deps/fuzzer.Tpo: Permission denied

Would someone be able to help we with the dbox-storage error?

Thanks,

James.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


doveadm sync failed after adding "mechanisms = plain login" to passdb

2024-01-30 Thread James via dovecot

Hello,

After adding `mechanisms = plain login` to passdb (passwd-file), 
`doveadm -D sync -u e...@example.com -d -N -l 30 -U` failed with

```
Jan 26 06:49:22 doveadm(e...@example.com): Error: 
remote(server2.example.com:61526): doveadm(1.1.1.1,e...@example.com): 
auth-master: passdb lookup(e...@example.com): Auth PASS lookup failed
Jan 26 06:49:22 doveadm(e...@example.com): Error: 
remote(server2.example.com:61526): doveadm(1.1.1.1,e...@example.com): 
/run/dovecot/auth-userdb: passdb lookup failed (to see if user is 
proxied, because doveadm_port is set)

```
With `auth_debug=yes`, I found these lines.
```
Jan 26 04:55:37 server2 dovecot: auth: Debug: skipping passdb: mechanism 
filtered
Jan 26 04:55:37 server2 dovecot: auth: Error: All password databases 
were skipped

```
What should I do to fix it without deleting `mechanisms = plain login`?
More information could be found on [this 
link](https://github.com/orgs/docker-mailserver/discussions/3836#discussioncomment-8262101).


Thanks
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: Setting up a new server

2023-10-30 Thread James

On 28/10/2023 07:31, James Cloos wrote:


Also I'd like to use imap-specific passwds for each user.  My tests so
far have used the login passwds for each user.  Not necessarily virtual
users, just imap-specific passwds.  There is already a pgsql server
handy; I take it that would be the way to go for passdb and userdb
lookups, yes?


Yes!  If you have the DB already you have done most of the work.  I use 
postgresql for dovecot.  I would urge you to use virtual users with the 
user name of the email address.  All my users are uid:gid vmail:vmail.


My mail database is used for other functions as well as dovecot but this 
cut down listing has the columns used by dovecot (excepting typos... 
keep asking).


I have a table for 'mailbox'.  It references table 'domain' but as it 
uses a text key the mailbox table will stand alone and no join on lookup.



mail=# \d mailbox
Table "public.mailbox"
   Column|  Type  | Collation | Nullable | Default
-++---+--+-
 username| character varying(255) |   | not null |
 allow_nets  | character varying(255) |   |  |
 imap| boolean|   |  |
 password| character varying(255) |   |  |
 pop3| boolean|   |  |
 maxstorage  | integer|   |  |
 realname| character varying(255) |   |  |
 sieve   | boolean|   |  |
 smtp| boolean|   |  |
 domain  | character varying(255) |   | not null |
 maxcount| integer|   |  |

Indexes:
"mailbox_pkey" PRIMARY KEY, btree (username, domain)
Foreign-key constraints:
"fk_mailbox_domain" FOREIGN KEY (domain) REFERENCES domain(name)



/etc/opt/.../dovecot-sql.conf has lines:


user_query = "SELECT 'vmail' AS uid, 'vmail' AS gid, allow_nets, 
'*:storage=' || maxstorage || 'M'  AS quota_rule, '*:messages=' || 
maxcount AS quota_rule2 FROM mailbox WHERE username = '%n' AND domain = 
'%d' AND smtp = true;"



password_query = "SELECT password, allow_nets, '*:storage=' || 
maxstorage || 'M' AS userdb_quota_rule, '*:messages=' || maxcount AS 
userdb_quota_rule2 FROM mailbox WHERE username = '%n' AND domain = '%d' 
AND %Ls = true;"



Take this as hints; consult the documentation.


James.


___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Setting up a new server

2023-10-28 Thread James Cloos
I'm in the process of switching to dovecot for my lan imap
and have a couple of questions.

I want to keep the folder names unchanged.  For mailing lists I use the
posting address (eg f...@bar.co); for others I use / as the separator.
If I use mdbox, will those choices cause any issues?  My impression is
not, but I'd like to be sure.

Also I'd like to use imap-specific passwds for each user.  My tests so
far have used the login passwds for each user.  Not necessarily virtual
users, just imap-specific passwds.  There is already a pgsql server
handy; I take it that would be the way to go for passdb and userdb
lookups, yes?

Thanks,

-JimC
-- 
James Cloos  OpenPGP: 0x997A9F17ED7DAEA6
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Bug - doveadm backup out-of-memory kill/crash when no separators set

2023-07-29 Thread James Harrison

Hi,

I hit a fun issue with doveadm when migrating from dovecot 2.2.36 
(1f10bfa63) to 2.3.19.1 (9b53102964) (CentOS 7 to Debian 12).


When running doveadm -v -D backup -R -u "user@name" tcp:localhost:1234, 
I found that the first sync would always work, but subsequent runs of 
the command would cause doveadm to reach a subfolder (Archives/2008 in 
the example below) and then silently mmap() increasing powers of 2 
before the OOM killer finally got it. Deleting the source folder caused 
it to remain stuck on the same folder, but for the deletion event.


I switched memory overcommit off to force a crash, and got a gdb backtrace:

#0  __pthread_kill_implementation (threadid=, 
signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x7786bd9f in __pthread_kill_internal (signo=6, 
threadid=) at ./nptl/pthread_kill.c:78
#2  0x7781cf32 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/posix/raise.c:26

#3  0x77807472 in __GI_abort () at ./stdlib/abort.c:79
#4  0x77b4ffae in default_fatal_finish (status=0, 
type=LOG_TYPE_PANIC) at ../lib/failures.c:465
#5  fatal_handler_real (ctx=, format=, 
args=) at ../lib/failures.c:477
#6  0x77bfa081 in default_fatal_handler (ctx=, 
format=, args=) at ../lib/failures.c:485
#7  0x77b5017c in i_panic (format=0x77c5d748 "data stack: 
Out of memory when allocating %zu bytes") at ../lib/failures.c:530
#8  0x77b4f67f in mem_block_alloc (min_size=min_size@entry=16) 
at ../lib/data-stack.c:386
#9  0x77bf8b60 in t_malloc_real (size=, 
permanent=) at ../lib/data-stack.c:492
#10 0x77c348f1 in t_strdup_until 
(start=start@entry=0x5565f440, end=end@entry=0x5565f448) at 
../lib/strfuncs.c:270
#11 0x555adc62 in convert_name_to_remote_sep 
(name=0x5565f440 "Archives/2008", tree=0x5564e188) at 
dsync/dsync-mailbox-tree.c:270
#12 dsync_mailbox_tree_build_name128_remotesep_hash 
(tree=0x5564e188) at dsync/dsync-mailbox-tree.c:315
#13 dsync_mailbox_tree_find_delete (tree=0x5564e188, 
del=0x556469f0) at dsync/dsync-mailbox-tree.c:405
#14 0x555a4195 in dsync_brain_mailbox_tree_add_delete 
(tree=0x5564e188, other_tree=0x5564f5f8, other_del=0x556469f0,
    node_r=0x7fffe350, status_r=0x7fffe348) at 
dsync/dsync-brain-mailbox-tree.c:504
#15 0x555a44cd in dsync_brain_recv_mailbox_tree_deletes 
(brain=0x5564b2d8) at dsync/dsync-brain-mailbox-tree.c:590
#16 0x555a5365 in dsync_brain_run_real 
(brain=brain@entry=0x5564b2d8, 
changed_r=changed_r@entry=0x7fffe453) at dsync/dsync-brain.c:709
#17 0x555a59f9 in dsync_brain_run (changed_r=0x7fffe453, 
brain=0x5564b2d8) at dsync/dsync-brain.c:752
#18 dsync_brain_run (changed_r=0x7fffe453, brain=0x5564b2d8) at 
dsync/dsync-brain.c:740

#19 dsync_brain_run_io (context=) at dsync/dsync-brain.c:113
#20 dsync_brain_run_io (context=0x5564b2d8) at dsync/dsync-brain.c:100
#21 0x555b23df in dsync_ibc_stream_input (ibc=0x55646720) at 
dsync/dsync-ibc-stream.c:232
#22 0x77c11cd9 in io_loop_call_io (io=0x556418f0) at 
../lib/ioloop.c:737
#23 0x77c13aa2 in io_loop_handler_run_internal 
(ioloop=ioloop@entry=0x5560bf40) at ../lib/ioloop-epoll.c:222
#24 0x77c13b50 in io_loop_handler_run 
(ioloop=ioloop@entry=0x5560bf40) at ../lib/ioloop.c:789
#25 0x77c13d10 in io_loop_run (ioloop=0x5560bf40) at 
../lib/ioloop.c:762
#26 0x5558b22e in cmd_dsync_run_remote (user=0x55637248) at 
./src/doveadm/doveadm-dsync.c:543
#27 cmd_dsync_run (_ctx=0x5561f288, user=0x55637248) at 
./src/doveadm/doveadm-dsync.c:750
#28 0x5558bd12 in doveadm_mail_next_user (ctx=0x5561f288, 
error_r=0x7fffe818) at ./src/doveadm/doveadm-mail.c:464
#29 0x5558cf45 in doveadm_mail_cmd_exec (wildcard_user=0x0, 
ctx=0x5561f288) at ./src/doveadm/doveadm-mail.c:659
#30 doveadm_cmd_ver2_to_mail_cmd_wrapper (cctx=0x7fffe970) at 
./src/doveadm/doveadm-mail.c:988
#31 0x55597622 in doveadm_cmd_run_ver2 (argc=5, 
argv=0x5560ba38, cctx=cctx@entry=0x7fffe970) at 
./src/doveadm/doveadm-cmd.c:465
#32 0x55597697 in doveadm_cmd_try_run_ver2 (cmd_name=out>, argc=, argv=, cctx=0x7fffe970)

    at ./src/doveadm/doveadm-cmd.c:363
#33 0x5557919a in main (argc=, argv=out>) at ./src/doveadm/doveadm.c:361


Neither server had a separator set in the inbox namespace. Having 
identified the convert_name_to_remote_sep function as the likely trigger 
point I set the separator to / on both ends, and doveadm backup now runs 
without issue.


Can provide config if helpful but I think probably irrelevant to this 
one - the namespace config is the default other than the separator item. 
May be worth mentioning that having the separator set can avoid this 
condition in the doveadm backup docs - this took me a while to fig

Re: "sh: doveadm: command not found" with doveadm -Dv backup -R

2023-02-28 Thread James Brown
> On 28 Feb 2023, at 5:53 pm, Aki Tuomi  wrote:
> 
>> On 28/02/2023 03:54 EET James Brown  wrote:
>> 
>> I’m moving from an old mail server to a new one.
>> 
>> On the new one, trying to transfer mailboxes from the old machine using 
>> doveadm. It fails saying it can’f find doveadm:
>> 
>> % sudo doveadm -Dv backup -R -u myu...@bordo.com.au remote:mail.bordo.com.au
>> Password:
>> Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm
>> ...
>> (r...@mail.bordo.com.au) Password:
>> sh: doveadm: command not found
>> Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): 
>> Error: read(mail.bordo.com.au) failed: EOF (version not received)
>> Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): 
>> Error: Remote command returned error 127: ssh mail.bordo.com.au doveadm 
>> dsync-server -umyu...@bordo.com.au -U
>> Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): 
>> Debug: User session is finished
>> Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): 
>> Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb 
>> (uid=0): Disconnected: Connection closed (fd=9)
>> 
>> Does this mean the remote machine can’t fine doveadm?
>> 
>> (Both machines are running macOS)
>> 
>> I could probably transfer the mailboxes using rsync, but I thought that 
>> doveadm was a better way.
>> 
>> Thanks,
>> 
>> James.
> 
> Yeah, it means it can't find doveadm on the remote machine.
> 
> Please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/
> 
> Aki

Thanks Aki, those instructions are great. All working now. Added the service 
doveadm and pw to old server and pw to new one and all works. And used 
tcp:oldmailserver:port instead of ‘remote:’ in command.

So sudo doveadm -Dv backup -R -u myu...@bordo.com.au tcp:mail.bordo.com.au:12354

Simple!

Thanks again,

James.

"sh: doveadm: command not found" with doveadm -Dv backup -R

2023-02-27 Thread James Brown
mailbox in list index: Mailbox not found
Feb 28 12:42:02 doveadm(myu...@bordo.com.au): Debug: Namespace : Using 
permissions from /usr/local/virtual/bordo.com.au/myuser: mode=0700 gid=default
(r...@mail.bordo.com.au) Password:
sh: doveadm: command not found
Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): 
Error: read(mail.bordo.com.au) failed: EOF (version not received)
Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): 
Error: Remote command returned error 127: ssh mail.bordo.com.au doveadm 
dsync-server -umyu...@bordo.com.au -U
Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): 
Debug: User session is finished
Feb 28 12:42:06 dsync-local(myu...@bordo.com.au): 
Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb (uid=0): 
Disconnected: Connection closed (fd=9)

Does this mean the remote machine can’t fine doveadm?

(Both machines are running macOS)

I could probably transfer the mailboxes using rsync, but I thought that doveadm 
was a better way.

Thanks,

James.

Re: Auth-worker, unknown scheme ARGON2ID

2023-02-22 Thread James Brown
On 22 Feb 2023, at 6:12 pm, Aki Tuomi  wrote:
> 
> You can use 
> 
> env PKG_CONFIG_PATH=/opt/homebrew/lib/ ./configure …
> 
> Aki


Aki was a tremendous help and got it working for me. For anyone else who wants 
to use Dovecot on macOS with the ARGON2I or ARGON2ID password schemes, these 
are the steps:

First, need to install pkg-config using Homebrew.

Then install libsodium from Homebrew. 

Then:

env PKG_CONFIG_PATH=/opt/homebrew/lib/pkgconfig pkg-config --libs libsodium  

CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include 
LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl 
--with-mysql --with-sodium

(Probably should have not linked to specific version of OpenSSL 3, but it works 
so I’m not changing it now!)

make

sudo make install

All should work, start Dovecot with 

sudo /usr/local/sbin/dovecot

And test with:

 % sudo doveadm pw -l   
 
SHA1 SSHA512 SCRAM-SHA-256 BLF-CRYPT PLAIN HMAC-MD5 OTP SHA512 SHA DES-CRYPT 
CRYPT SSHA MD5-CRYPT PLAIN-MD4 PLAIN-MD5 SCRAM-SHA-1 CLEAR CLEARTEXT ARGON2I 
ARGON2ID SSHA256 MD5 PBKDF2 SHA256 CRAM-MD5 PLAIN-TRUNC SMD5 DIGEST-MD5 
LDAP-MD5 

Hope that saves someone some time and frustration.

Probably future me! :-)

Huge thanks to Aki for his help and patience,

James.

Re: Auth-worker, unknown scheme ARGON2ID

2023-02-21 Thread James Brown
On 22 Feb 2023, at 5:53 pm, Aki Tuomi  wrote:
> 
>>>> 
>>>> % locate libsodium
>>>> /opt/homebrew/Cellar/libsodium
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew/libsodium.rb
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/AUTHORS
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/ChangeLog
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/INSTALL_RECEIPT.json
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/LICENSE
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/README.markdown
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/core.h
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/crypto_aead_aes256gcm.h
>>>> ...
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/utils.h
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/version.h
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium.h
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.23.dylib
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.a
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig
>>>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig/libsodium.pc
>>>> /opt/homebrew/lib/libsodium.23.dylib
>>>> /opt/homebrew/lib/libsodium.a
>>>> /opt/homebrew/lib/libsodium.dylib
>>>> /opt/homebrew/lib/pkgconfig/libsodium.pc
>>>> /opt/homebrew/opt/libsodium
>>>> /opt/homebrew/var/homebrew/linked/libsodium
>>>> 
>>>> What do I need to use to stop the:Can't build with libsodium: not found
>>>> 
>>>> Sorry these are such stupid questions.
>>>> 
>>>> James.
> 
> Dovecot uses pkg-config to find out about libsodium, and it seems you're 
> missing libsodium.pc in whatever place mac uses to hide them.
> 
> Maybe the .pc file is in the homebrew directory somewhere and needs to be 
> copied, or you can make it yourself. Make sure to change the version and 
> paths.
> 
> prefix=/usr
> exec_prefix=${prefix}
> libdir=${prefix}/lib/x86_64-linux-gnu
> includedir=${prefix}/include
> 
> Name: libsodium
> Version: 1.0.18
> Description: A modern and easy-to-use crypto library
> 
> Libs: -L${libdir} -lsodium
> Libs.private:  -pthread 
> Cflags: -I${includedir}
> 
> Aki

Getting somewhere - thanks Aki.

% locate libsodium.pc
/opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig/libsodium.pc
/opt/homebrew/lib/pkgconfig/libsodium.pc
/usr/local/lib/pkgconfig/libsodium.pc

Looking at /opt/homebrew/lib/pkgconfig/libsodium.pc it is:

prefix=/opt/homebrew/Cellar/libsodium/1.0.18_1
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: libsodium
Version: 1.0.18
Description: A modern and easy-to-use crypto library

Libs: -L${libdir} -lsodium
Libs.private:  -pthread
Cflags: -I${includedir}

The version of libsodium I downloaded and installed from source is:

prefix=/usr/local
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: libsodium
Version: 1.0.18
Description: A modern and easy-to-use crypto library

Libs: -L${libdir} -lsodium
Libs.private: -lpthread -pthread
Cflags: -I${includedir}

Do I need to edit one of these files, or somehow tell .configure to look at 
/opt/homebrew/lib/ ?

James.



Re: Auth-worker, unknown scheme ARGON2ID

2023-02-21 Thread James Brown
> On 22 Feb 2023, at 5:33 pm, Aki Tuomi  wrote:
>> 
>> Thanks Aki, that was helpful. When I add that I get:
>> 
>> checking for LIBSODIUM... no
>> configure: error: Can't build with libsodium: not found
>> 
>> So I have to tell it where libsodium is.
>> 
>> Tried:
>> 
>> CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include 
>> LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib 
>> LIBSODIUM_LIBS=-L/opt/homebrew/Cellar/libsodium ./configure 
>> --with-ssl=openssl --with-mysql —with-sodium
>> 
>> But that gives:
>> 
>> configure: WARNING: Libtool does not cope well with whitespace in `pwd`
>> checking build system type... Invalid configuration `—-with-sodium': machine 
>> `—-with' not recognized
>> 
>> Location of libsodium:
>> 
>> % locate libsodium
>> /opt/homebrew/Cellar/libsodium
>> /opt/homebrew/Cellar/libsodium/1.0.18_1
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/.brew/libsodium.rb
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/AUTHORS
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/ChangeLog
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/INSTALL_RECEIPT.json
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/LICENSE
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/README.markdown
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/core.h
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/crypto_aead_aes256gcm.h
>> ...
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/utils.h
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/version.h
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium.h
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.23.dylib
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.a
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig
>> /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig/libsodium.pc
>> /opt/homebrew/lib/libsodium.23.dylib
>> /opt/homebrew/lib/libsodium.a
>> /opt/homebrew/lib/libsodium.dylib
>> /opt/homebrew/lib/pkgconfig/libsodium.pc
>> /opt/homebrew/opt/libsodium
>> /opt/homebrew/var/homebrew/linked/libsodium
>> 
>> What do I need to use to stop the:Can't build with libsodium: not found
>> 
>> Sorry these are such stupid questions.
>> 
>> James.
> 
> You accidentically used a UTF-8 special dash instead of two normal dashes.
> 
> Aki
> 

Yes, I think I picked that up. I think I might have typed it into the email and 
it changed it (I’ve run the configure command so many times.)

CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include 
LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib 
LIBSODIUM_LIBS=-L/opt/homebrew/Cellar/libsodium ./configure --with-ssl=openssl 
--with-mysql --with-sodium

Fails with:

checking for mysql_ssl_set in -lmysqlclient... yes
checking for LIBSODIUM... no
configure: error: Can't build with libsodium: not found

This is so frustrating, but I’ll be jumping for joy if we can get it to work.

James.

Re: Auth-worker, unknown scheme ARGON2ID

2023-02-21 Thread James Brown
> On 22 Feb 2023, at 4:14 pm, Aki Tuomi  wrote:
> 
>> I’ve spent ages on this and am getting really desperate! :-(
>> 
>> CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include 
>> LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl 
>> --with-mysql
>> ...
>> Install prefix . : /usr/local
>> File offsets ... : 64bit
>> I/O polling  : kqueue
>> I/O notifys  : kqueue
>> SSL  : yes (OpenSSL)
>> GSSAPI . : no
>> passdbs  : static passwd passwd-file pam checkpassword sql
>> : -shadow -bsdauth -ldap
>> userdbs  : static prefetch passwd passwd-file checkpassword sql
>> 
>> sudo doveadm pw -l
>> SHA1 SSHA512 SCRAM-SHA-256 BLF-CRYPT PLAIN HMAC-MD5 OTP SHA512 SHA DES-CRYPT 
>> CRYPT SSHA MD5-CRYPT PLAIN-MD4 PLAIN-MD5 SCRAM-SHA-1 CLEAR CLEARTEXT SSHA256 
>> MD5 PBKDF2 SHA256 CRAM-MD5 PLAIN-TRUNC SMD5 DIGEST-MD5 LDAP-MD5
>> 
>> How do I get ARGON2I, ARGON2ID in that list?
>> 
>> Has anybody got Dovecot to work on recent macOS with these password schemes? 
>> Any hints?
>> 
>> Thanks, James.
> 
> You need to use --with-sodium when building.
> 
> Aki

Thanks Aki, that was helpful. When I add that I get:

checking for LIBSODIUM... no
configure: error: Can't build with libsodium: not found

So I have to tell it where libsodium is.

Tried:

CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include 
LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib 
LIBSODIUM_LIBS=-L/opt/homebrew/Cellar/libsodium ./configure --with-ssl=openssl 
--with-mysql —with-sodium

But that gives:

configure: WARNING: Libtool does not cope well with whitespace in `pwd`
checking build system type... Invalid configuration `—-with-sodium': machine 
`—-with' not recognized

Location of libsodium:

 % locate libsodium
/opt/homebrew/Cellar/libsodium
/opt/homebrew/Cellar/libsodium/1.0.18_1
/opt/homebrew/Cellar/libsodium/1.0.18_1/.brew
/opt/homebrew/Cellar/libsodium/1.0.18_1/.brew/libsodium.rb
/opt/homebrew/Cellar/libsodium/1.0.18_1/AUTHORS
/opt/homebrew/Cellar/libsodium/1.0.18_1/ChangeLog
/opt/homebrew/Cellar/libsodium/1.0.18_1/INSTALL_RECEIPT.json
/opt/homebrew/Cellar/libsodium/1.0.18_1/LICENSE
/opt/homebrew/Cellar/libsodium/1.0.18_1/README.markdown
/opt/homebrew/Cellar/libsodium/1.0.18_1/include
/opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium
/opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/core.h
/opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/crypto_aead_aes256gcm.h
...
/opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/utils.h
/opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium/version.h
/opt/homebrew/Cellar/libsodium/1.0.18_1/include/sodium.h
/opt/homebrew/Cellar/libsodium/1.0.18_1/lib
/opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.23.dylib
/opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.a
/opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib
/opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig
/opt/homebrew/Cellar/libsodium/1.0.18_1/lib/pkgconfig/libsodium.pc
/opt/homebrew/lib/libsodium.23.dylib
/opt/homebrew/lib/libsodium.a
/opt/homebrew/lib/libsodium.dylib
/opt/homebrew/lib/pkgconfig/libsodium.pc
/opt/homebrew/opt/libsodium
/opt/homebrew/var/homebrew/linked/libsodium

What do I need to use to stop the: Can't build with libsodium: not found

Sorry these are such stupid questions. 

James.

Re: Auth-worker, unknown scheme ARGON2ID

2023-02-21 Thread James Brown
On 21 Feb 2023, at 10:12 pm, James Brown  wrote:
> 
> The new one has Dovecot compiled with same configure options, same 
> configuration files, but fails to authenticate:
> 
> Feb 21 21:51:03 master: Info: Dovecot v2.3.20 (80a5ac675d) starting up for 
> imap, pop3 (core dumps disabled)
> Feb 21 21:51:33 auth-worker(11701): Error: conn unix:auth-worker (uid=214): 
> auth-worker<1>: 
> sql(jlbr...@bordo.com.au,::1,): Unknown 
> scheme ARGON2ID
> 
> ./configure --with-ssl=openssl --with-mysql
> 
> Was what I used to compile.
> 
> Using OpenSSL 3.0.8. New server is running macOS 13.2.1 Ventura.
> 
> (Have also installed libsodium 1.0.18_1)
> 
> Any help would be most appreciated.
> 
> thanks,
> 
> James.


I’ve spent ages on this and am getting really desperate! :-(

CPPFLAGS=-I/opt/homebrew/Cellar/openssl@3/3.0.8/include 
LDFLAGS=-L/opt/homebrew/opt/openssl@3/lib ./configure --with-ssl=openssl 
--with-mysql

...
Install prefix . : /usr/local
File offsets ... : 64bit
I/O polling  : kqueue
I/O notifys  : kqueue
SSL  : yes (OpenSSL)
GSSAPI . : no
passdbs  : static passwd passwd-file pam checkpassword sql
 : -shadow -bsdauth -ldap
userdbs  : static prefetch passwd passwd-file checkpassword sql


sudo doveadm pw -l  
SHA1 SSHA512 SCRAM-SHA-256 BLF-CRYPT PLAIN HMAC-MD5 OTP SHA512 SHA DES-CRYPT 
CRYPT SSHA MD5-CRYPT PLAIN-MD4 PLAIN-MD5 SCRAM-SHA-1 CLEAR CLEARTEXT SSHA256 
MD5 PBKDF2 SHA256 CRAM-MD5 PLAIN-TRUNC SMD5 DIGEST-MD5 LDAP-MD5 

How do I get ARGON2I, ARGON2ID in that list?

Has anybody got Dovecot to work on recent macOS with these password schemes? 
Any hints?

Thanks, James.

Auth-worker, unknown scheme ARGON2ID

2023-02-21 Thread James Brown
On my current mail server I was able to upgrade everyone’s password scheme to 
ARGON2ID with no issues, I just had to have:
service auth {
   vsz_limit = 0
}
This was an older version of macOS than the new server I’m trying to migrate to.

The new one has Dovecot compiled with same configure options, same 
configuration files, but fails to authenticate:

Feb 21 21:51:03 master: Info: Dovecot v2.3.20 (80a5ac675d) starting up for 
imap, pop3 (core dumps disabled)
Feb 21 21:51:33 auth-worker(11701): Error: conn unix:auth-worker (uid=214): 
auth-worker<1>: 
sql(jlbr...@bordo.com.au,::1,): Unknown scheme 
ARGON2ID

./configure --with-ssl=openssl --with-mysql

Was what I used to compile.

Using OpenSSL 3.0.8. New server is running macOS 13.2.1 Ventura.

(Have also installed libsodium 1.0.18_1)

Any help would be most appreciated.

thanks,

James.


service(imap-login): Fatal: setrlimit(RLIMIT_DATA, 268435456): Invalid argument macOS

2023-02-20 Thread James Brown
I fixed the issue with Dovecot not starting on macOS Ventura by adding this to 
10-master.conf:

service log {
  vsz_limit = 0
}

But now when I try to login via Imap on port 143 I get the same error for 
imap-login:

Feb 20 15:55:41 imap-login: Error: Feb 20 15:55:41 service(imap-login): Fatal: 
setrlimit(RLIMIT_DATA, 268435456): Invalid argument
Feb 20 15:55:41 imap-login: Fatal: master: service(imap-login): child 11781 
returned error 89 (Fatal failure)
Feb 20 15:55:41 master: Error: service(imap-login): command startup failed, 
throttling for 60.000 secs
Feb 20 16:13:04 imap-login: Error: Feb 20 16:13:04 service(imap-login): Fatal: 
setrlimit(RLIMIT_DATA, 268435456): Invalid argument
Feb 20 16:13:04 imap-login: Fatal: master: service(imap-login): child 20370 
returned error 89 (Fatal failure)
Feb 20 16:13:04 master: Error: service(imap-login): command startup failed, 
throttling for 60.000 secs

Config:

sudo /usr/local/sbin/dovecot -n
# 2.3.20 (80a5ac675d): /usr/local/etc/dovecot/dovecot.conf
# OS: Darwin 22.3.0 arm64  apfs
# Hostname: mail-server-mac-studio.local
auth_mechanisms = plain login
auth_verbose = yes
auth_verbose_passwords = yes
default_internal_group = mail
default_internal_user = _dovecot
default_login_user = _dovenull
first_valid_gid = 0
first_valid_uid = 102
last_valid_gid = 501
last_valid_uid = 105
log_path = /var/log/dovecot.log
mail_gid = 102
mail_location = maildir:/usr/local/virtual/%d/%u
mail_uid = 102
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix = 
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
protocols = imap pop3
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = _postfix
mode = 0660
user = _postfix
  }
  unix_listener auth-userdb {
mode = 0660
user = _vmail
  }
}
service imap-login {
  vsz_limit = 0
}
service log {
  vsz_limit = 0
}
ssl_cert = 

Re: SOLVED: Re: Dovecot does not start on MacOS 12.01, and now 13.2.1 - RLIMIT_DATA

2023-02-17 Thread James Brown
Sorry, replying to an old thread found on mail-archive.com 
<http://mail-archive.com/>, so it’s not threaded email.

https://www.mail-archive.com/dovecot@dovecot.org/msg84776.html


I have just installed Dovecot onto Mac Studio running Ventura (macOS 13.2.1) 
and get the same error with Dovecot 2.3.20:

service(log): Fatal: setrlimit(RLIMIT_DATA, 268435456): Invalid argument

Adding the ’service log’ stanza to 10-master.conf fixed the issue.

Is this still the correct way to handle it?

If so it would be great to have it documented somewhere. Just lucky I found the 
above mail thread.

Thanks,

James.



Hi!

That solution is bit overly broad way to solve this, I would instead recommend
service log {
  vsz_limit = 0
}

so that only service log is affected.

Aki

> On 31/10/2021 20:36 Don Feliciano  wrote:
> 
> 
> 
> A kind person who isn’t on the mailing list share the solution to this with 
> me:
> 
> default_vsz_limit = 0
> 
> 
> > On Oct 28, 2021, at 12:12 PM, Don Feliciano  
> > wrote:
> > 
> >
> > Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 
> > 1000), because of default_client_limit
> > Oct 28 12:08:48 service(log): Fatal: setrlimit(RLIMIT_DATA, 268435456): 
> > Invalid argument
> > 
> > 
> > > On Oct 27, 2021, at 11:08 AM, Don Feliciano  
> > > wrote:
> > > 
> > > I've been happily running Dovecot on my Mac for many years (installed via 
> > > Homebrew). After upgrading to Monterey (MacOS 12.01), it no longer starts:
> > > 
> > > $ sw_vers
> > > ProductName: macOS
> > > ProductVersion: 12.0.1
> > > BuildVersion: 21A559
> > > 
> > > $ uname -a
> > > Darwin dfelicia-mac 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 
> > > 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64 x86_64
> > > 
> > > $ sudo dovecot -F
> > > Oct 27 10:11:18 service(log): Fatal: setrlimit(RLIMIT_DATA, 268435456): 
> > > Invalid argument



Permissions issue with doveadm mailbox command

2022-12-30 Thread James Moe


I have been exploring dovecot and came across this:

$ sudo doveadm mailbox list
doveadm(root): Error: Mail access for users with UID 303 not permitted (see
first_valid_uid in config file, uid from mail_uid setting).

  Hmm. After setting "first_valid_uid = 300":

$ sudo doveadm mailbox list
doveadm(root)<4380><4VP8EENer2McEQAAvbJltg>: Error: chdir(/root/) failed:
Permission denied (euid=303(vmail) egid=303(vmail) missing +x perm: /root, dir
owned by 0:0 mode=0700)
INBOX

  I do not understand the error message.

  The current configuration. (Note I have changed first_valid_uid to its
original value.)
$ postconf -n
# 2.3.15 (0503334ab1): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.15 (e6a84e31)
# OS: Linux 5.14.21-150400.24.33-default x86_64  btrfs
# Hostname: sma-server3.sma.com
auth_mechanisms = plain login
base_dir = /var/run/dovecot/
debug_log_path = /data01/var/log/dovecot/dovecot-debug.log
disable_plaintext_auth = no
first_valid_uid = 1001
info_log_path = /data01/var/log/dovecot/dovecot-info.log
log_path = /data01/var/log/dovecot/dovecot.log
log_timestamp = %Y-%m-%dT%H:%M:%S
mail_gid = vmail
mail_location = maildir:/var/mail/vmail/%u/
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
mime foreverypart extracttext
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
passdb {
  driver = pam
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  sieve = file:~/sieve;active=~/.dovecot.sieve
}
protocols = imap
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
  }
}
service log {
  group = users
  user = dovecot
}
ssl = no
ssl_cipher_list =
ALL:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH
ssl_options = no_compression
ssl_prefer_server_ciphers = yes
userdb {
  driver = passwd
}
userdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}


-- 
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.


Re: [SOLVED] Permissions for dovecot logging

2022-12-30 Thread James Moe
On 2022-12-27 16:19, James Moe wrote:

>   I changed logging to use a path rather than syslog. Doing so makes it easier
> to work with fail2ban.
>   Dovecot fails to start with the error:
> Can't open log file /data01/var/log/dovecot.log: Permission denied
> 
  Yes, it was apparmor. It has been enabled for a couple of months. Dovecot is
the first app that I've added that has a apparmor profile. After adding the
necessary entry to the profile, logging proceeded as expected.

In  I added:
  owner /data01/var/log/dovecot/* a,


-- 
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.



Re: Permissions for dovecot logging

2022-12-29 Thread James Moe
On 2022-12-28 00:27, Aki Tuomi wrote:

> The `log` service runs by default as root, not as dovecot. 
>
  Then I do not understand why there is a permissions problem at all. It is 
root!

> If data01 is a NFS mount, then root may become squashed.
> 
  Not an NFS mount. It is local.

> If you want to run log as `dovecot`, you can do so with
> 
> service log {
>   user = dovecot
> }
>
  Permission is still denied.
  Where do I find information about "status=80/n/a"?

  I did not include all two of the syslog entries in the previous message:
2022-12-29T20:17:56-0700 sma-server3 dovecot[12102]: Can't open log file
/data01/var/log/dovecot.log: Permission denied
2022-12-29T20:17:56-0700 sma-server3 systemd[1]: dovecot.service: Main process
exited, code=exited, status=80/n/a


-- 
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.



Permissions for dovecot logging

2022-12-27 Thread James Moe
dovecot 2.3.15
opensuse LEAP 15.4

  I changed logging to use a path rather than syslog. Doing so makes it easier
to work with fail2ban.
  Dovecot fails to start with the error:
Can't open log file /data01/var/log/dovecot.log: Permission denied

  Permissions:
drwxrwxr-x 1 root   users 104 Feb 25  2018 /data01/
drwxrwxr-x 1 sma-user3x users 102 Dec 17 14:50 /data01/var/
drwxrwxr-x 1 sma-user3x users 146 Dec 27 15:37 /data01/var/log/
drwxrwxr-x 1 dovecotusers  22 Dec 27 15:47 /data01/var/log/dovecot/

  "dovecot" is a member of "users".

  What "permission" am I missing?

  Note: A long time ago I had a problem with programs consuming all available
space on the system disk with log or backup files. I have since gotten in the
habit of putting log files on a non-system disk.

-- 
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.


Re: What do I do with the ".ext?"

2022-12-18 Thread James Moe
On 2022-12-17 00:09, Aki Tuomi wrote:

> Please see 
> https://doc.dovecot.org/configuration_manual/authentication/sql/#authentication-sql
>  how to use these.
>
  Thank you!

-- 
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.



What do I do with the ".ext?"

2022-12-16 Thread James Moe
dovecot 2.3.15
opensuse LEAP 15.4

  I am installing dovecot and have postfix as the SMTP agent. It also needs to
use a MySQL database. In  there is the main dovecot.conf file, and
a number of other files with the extension ".conf.ext."
  It is not clear how to use these. I could nothing in the documentation that
discusses it.
  Do I copy one, say dovecot-sql.conf.ext, modify it:
- rename the copy without the ".ext?"
- or move the copy to the "conf.d" directory, with or without the "ext?"

-- 
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.


Re: SQL user_query quota_rule for *:messages=

2022-05-23 Thread James

On 23/05/2022 08:18, Aki Tuomi wrote:


Is it possible to override the message count limit with an SQL
user_query (and password_query)?




I tried returning userdb_quota_rule2 in the SQL but it did not work.
"SELECT ... '*:messages=' || maxcount AS userdb_quota_rule2 ...;"


Ref:
https://doc.dovecot.org/configuration_manual/authentication/user_database_extra_fields/#authentication-user-database-extra-fields


If you are returning that from userdb, you should not return it with `userdb_` 
prefix.


Thank you, correct advice, my mistake.  I have a combined password_query 
and made the mistake of adding userdb_ prefix in both.


This has enabled the user limit for messages (as well as storage) and I 
can see over messages count causes defer like for over storage. 
Progress!  Thank you for the help.



For reference the quota warning for messages is also working (from 
dovecot.conf):

  quota_warning6 = messages=90%% quota-warning 90 %u
Testing this was my objective which required setting a very low per user 
value for a test account.  Thank you.





SQL user_query quota_rule for *:messages=

2022-05-20 Thread James

Hello,

dovecot.conf sets the system default quota for storage and message count:

plugin {
   ...
  quota_rule = *:storage=1G
  quota_rule2 = *:messages=1
   ...
}

This quota_rule for the storage is overridden by an SQL user_query in 
dovecot-sql.conf:


user_query = "SELECT ... '*:storage=' || maxstorage || 'M'  AS 
userdb_quota_rule, ...;"


Is it possible to override the message count limit with an SQL 
user_query (and password_query)?





I tried returning userdb_quota_rule2 in the SQL but it did not work. 
"SELECT ... '*:messages=' || maxcount AS userdb_quota_rule2 ...;"



Ref:
https://doc.dovecot.org/configuration_manual/authentication/user_database_extra_fields/#authentication-user-database-extra-fields



Re: /usr/libexec/dovecot/anvil crashes immediately

2022-02-21 Thread James

On 08/02/2022 20:04, Friedrich Kink wrote:


I built a dovecot package for openindiana (which is a Solaris
derivative) from latest version 2.3.18. Everything compiles and builds
fine without any issue. Even subsequent installation and startup of main
dovecot process works as expected. But execution of
/usr/libexec/dovecot/anvil immediately crashes.


I can't see what is wrong from your trace.  You might get more help on 
the OpenIndiana mailing list.


I run dovecot on OmniOS (also a Solaris / illuminos derivative) without 
problem.  I wonder why you are running dovecot on openindiana which is 
focused for desktop use; why not use OmniOS for a server?  Your choice 
but you could run OpenIndiana global with a bhyve zone running OmniOS.



James.


Re: ZFS storage and backup

2021-11-19 Thread James

On 15/11/2021 16:18, infoomatic wrote:

Regarding
storage I tend to use sdbox, from what I have read it seems to be the
better option when using a COW filesystem compared to mdbox. One more


https://doc.dovecot.org/admin_manual/mailbox_formats/
sdbox   single-dbox, one message per file.
mdbox   multi-dbox, multiple messages per file.

so I guess sdbox is better with ZFS.  I could test each but I think I 
will find the IO used by dovecot is low for each.  I have one user with 
32,164 emails in INBOX and IO is not a problem.




question is: compression at file system level or in dovecot storage?


System.  The OS compresses using all CPUs in a separate process. - does 
dovecot?  Dovecot is smaller and simpler (--with-zlib=no etc).  You can 
change the ZFS compression anytime.  Text files remain plain text files 
even though they are compressed on disc.


When available, zstd in ZFS should be a better option than gzip.



The reason I am not sure to switch to ssds is that most servers are for
non-profit organisations, sports clubs etc. - they also need some
storage for pictures, their budget is quite low (so performance testing
would only be done out of my interest), and if spinning rust with
optimized settings suffices why not.


As you have the HDDs already wait until there is a problem before fixing 
it.  Over the internet I doubt anyone will notice and more importantly 
care enough to pay.  Your HDDs might be old and about to fail so other 
factors rise in importance.  Data security and continuity of service are 
more important than latency.


Do you have enough RAM for read cache?  A separate log for writes? L2ARC 
will only help if you have more active data than fits in RAM.



James



Re: ZFS storage and backup

2021-11-15 Thread James

On 14/11/2021 14:14, infoomatic wrote:

My setups are nothing special with few users, however, I would like to
have a nice setup, maybe some of you could contribute to this thread. We
are using slow spinning disks, but we may consider using ssds in a
not-so-distant future.

*) storages: any infos on ZFS options or whether to use mdbox or sdbox,
and what configs/options regarding compression etc.


OmniOS with ZFS here.

I use maildir - just a personal choice and inertia, I have no 
performance data, no problem and no reason to change.  I like being able 
to see emails as plain files.


zfs set compress=gzip

and no other changes from default, oh and atime=off on the whole 
machine.  Email gzips well, most other ZFSes I leave on lz4.  I say it 
is better to use the file system to compress rather than getting dovecot 
to do it.


$ zfs get compress,compressratio,used ...
NAME  PROPERTY   VALUE SOURCE
.//vmail  compressiongzip  received
.//vmail  compressratio  1.82x -
.//vmail  used   8.55G -

25 mailbox users ("nothing special with few users").

I moved the storage from HDD (mirror plus log) to SSD (mirror) and no 
one noticed, not even me knowing it had been done and over a local 
network.  I have enough RAM such that repeated reads are cached.


I will use native ZFS encryption soon.  I see no performance issues in test.

Don't get hung up on ZFS tuning, mostly ZFS just works.



*) backup: what is a best practice regarding backups? - using only the
dovecot tools or leveraging the great features of ZFS (or both) with
snapshots etc.?


I use automated snapshots and zfs send/receive to a remote backup 
machine.  I auto copy many ZFSes this way so it is minimal effort to do 
email too.


James


Logging to remote server

2021-08-26 Thread James Brown
My Google searches must be off, I couldn’t see how to do this.

Is there a way to log to a local file, but also send to a remote server?

Eg:

Log_path = /var/log/dovecot.log, @mylogserver.example.com:514

Thanks,

James

Re: Different userdb per inet_listener

2021-07-28 Thread James

On 28/07/2021 09:12, Miloslav Hůla wrote:

Now we would like to disable authentication for Postfix (SMTP), but
allow it for Dovecot (IMAP & POP3). Something like "receive-only".

Is there any way we can configure different passdb for mentioned
inet_listener?

Or is there any variable with "auth requetor name" we can use in SQL
query to differentiate the result?


%s for service

https://doc.dovecot.org/configuration_manual/config_file/config_variables/


Something like:

password_query = "SELECT password, allow_nets, '*:storage=' || quota || 
'M' AS userdb_quota_rule FROM mailbox WHERE username = '%n' AND domain = 
'%d' AND %Ls = true;"


Note the "AND %Ls = true".  The 'L' is for lower case.
Add boolean columns for the services to your database.




Re: Different userdb per inet_listener

2021-07-28 Thread James

On 28/07/2021 09:12, Miloslav Hůla wrote:

Now we would like to disable authentication for Postfix (SMTP), but
allow it for Dovecot (IMAP & POP3). Something like "receive-only".

Is there any way we can configure different passdb for mentioned
inet_listener?

Or is there any variable with "auth requetor name" we can use in SQL
query to differentiate the result?


%s for service

https://doc.dovecot.org/configuration_manual/config_file/config_variables/


Something like:

password_query = "SELECT password, allow_nets, '*:storage=' || quota || 
'M' AS userdb_quota_rule FROM mailbox WHERE username = '%n' AND domain = 
'%d' AND %Ls = true;"


Note the "AND %Ls = true".  The 'L' is for lower case.
Add boolean columns for the services to your database.




Re: [EXTERNAL] Sv: function for whitelisting IPs

2021-07-15 Thread James

On 15/07/2021 12:05, White, Daniel E. (GSFC-770.0)[NICS] wrote:


The custom login script -- in Dovecot or Roundcube or … ?
Is there any documentation for such scripting ?


https://doc.dovecot.org/configuration_manual/authentication/auth_policy/

It uses an http interface so it is easy to implement with existing http 
toolkits.  I wrote my own policy server in Java Jakarta EE9 because I 
can.  You might prefer an existing policy server or write your own in 
your favourite http implementation language.




Re: libdict_lua linking issues

2021-06-22 Thread James

On 22/06/2021 12:30, Timo Sirainen wrote:

libtool: link: gcc -std=gnu99 -m64 -march=x86-64 -fPIC -Os -Wall -W 
-Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts 
-Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -m64 
-o test-dict test-dict.o  ./.libs/libdict.a ../lib-test/.libs/libtest.a 
../lib/.libs/liblib.a -lsocket -lnsl -lsendfile
gcc: error: ./.libs/libdict_lua.a: No such file or directory
gmake[3]: *** [Makefile:630: test-dict-client] Error 1


Attached patch should work? You'll need to run autogen.sh again.


Patching src/lib-dict/Makefile.in did the job.  I don't know what is 
wrong with autoconf and automake - obviously I need a suite of tools to 
enable portability of autoconf, automake and libtool.




Re: libdict_lua linking issues

2021-06-22 Thread James

On 22/06/2021 12:30, Timo Sirainen wrote:


And on OmniOS / Solaris it failed with:

libtool: link: gcc -std=gnu99 -m64 -march=x86-64 -fPIC -Os -Wall -W 
-Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts 
-Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -m64 
-o test-dict test-dict.o  ./.libs/libdict.a ../lib-test/.libs/libtest.a 
../lib/.libs/liblib.a -lsocket -lnsl -lsendfile
gcc: error: ./.libs/libdict_lua.a: No such file or directory
gmake[3]: *** [Makefile:630: test-dict-client] Error 1

Attached patch should work? You'll need to run autogen.sh again.



No, similar error, I am slowly investigating.


I didn't run autogen.sh in the first place so I can't run it again. 
Running for the first time it moans about missing libtool:



Warning:  libtoolize does not appear to be available.  This means that
the automatic build preparation via autoreconf will probably not work.
Preparing the build by running each step individually, however, should
work and will be done automatically for you if autoreconf fails.

ERROR: Unable to locate GNU Libtool.

ERROR:  To prepare the Dovecot build system from scratch,
at least version 1.4.2 of GNU Libtool must be installed.







Re: [Dovecot-news] Dovecot v2.3.15 released

2021-06-21 Thread James

On 21/06/2021 17:39, Daniel J. Luke wrote:

On Jun 21, 2021, at 7:20 AM, Timo Sirainen  wrote:

Here's a new release with some security fixes and quite a lot of other changes 
as well.

 * Removed support for Lua 5.2. Use version 5.1 or 5.3 instead.

Looks like it doesn't want to build w/o lua now.

On my MacOS system configure says:


And on OmniOS / Solaris it failed with:

libtool: link: gcc -std=gnu99 -m64 -march=x86-64 -fPIC -Os -Wall -W 
-Wmissing-prototypes -Wmissing-declarations -Wpointer-arith 
-Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime 
-Wstrict-aliasing=2 -m64 -o test-dict test-dict.o  ./.libs/libdict.a 
../lib-test/.libs/libtest.a ../lib/.libs/liblib.a -lsocket -lnsl -lsendfile

gcc: error: ./.libs/libdict_lua.a: No such file or directory
gmake[3]: *** [Makefile:630: test-dict-client] Error 1


I confirm v2.3.15 has an error.

I tried combinations of --with-lua=no, --without-lua, etc, and also 
compiled lua and still it failed.














Re: NFS Locking Question

2021-05-28 Thread James

On 25/05/2021 16:46, Zane Zak wrote:

We have a single exim server that delivers mail to mbox files in /var/mail. We 
have a separate
server running dovecot that reads mail from these mbox files.

...

Is there a configuration change that needs to be made on the dovecot


I don't know about the NFS problem but change your configuration to use 
LMTP delivery.


https://wiki.dovecot.org/LMTP/Exim

Plus there are plenty of reasons to not use mbox.


Re: What imap ssl/auth settings work best with MS Outlook?

2021-04-30 Thread James

On 30/04/2021 08:13, @lbutlr wrote:


When you enter your email address, it would be TRIVIAL to check the MX records 
for the domain and fill those in for the SMTP and IMAP servers, allowing users 
to more easily add (if needed) the domain prefix.


Better to use DNS SVR records than guess from MX or domain.  I provide 
email SVRs but does any mail client use them?


https://tools.ietf.org/html/rfc6186


There is config-v1.1.xml, again I do not know which clients use, hence 
what I should provide, maybe I carry on providing as many methods as I can.




Re: Dovecot Replication Errors (only) when using tcps: as the mail_replica Protocol

2020-11-19 Thread James Pattinson



On 18/11/2020 19:37, Aakash Patel wrote:

Hello,

I have two mail servers and am also experiencing sporadic replication 
errors over tcps, similar to Reuben. Each server is running Dovecot 
2.3.11.3 (502c39af9) on Debian 10.6.


*Log entries from MX1*
Nov 18 00:39:26 mx1 dovecot: 
dsync-local(u...@example.com): Error: 
dsync(mx2.example.com): I/O has stalled, no activity for 600 seconds 
(last sent=mailbox, last recv=mailbox_state)
Nov 18 00:39:26 mx1 dovecot: 
dsync-local(u...@example.com): Error: Timeout 
during state=sync_mails (send=mailbox recv=mailbox)
Nov 18 06:39:32 mx1 dovecot: 
dsync-local(u...@example.com)<6bScGpwFtV+vEQAAPHKnuQ>: Error: 
dsync(mx2.example.com): I/O has stalled, no activity for 600 seconds 
(last sent=mailbox, last recv=mailbox_state)
Nov 18 06:39:32 mx1 dovecot: 
dsync-local(u...@example.com)<6bScGpwFtV+vEQAAPHKnuQ>: Error: Timeout 
during state=sync_mails (send=mailbox recv=mailbox)

*End*

*Log entries from MX2*
Nov 18 00:29:55 mx2 dovecot: 
dsync-local(u...@example.com): Error: Couldn't 
lock /var/vmail/u...@example.com/.dovecot-sync.lock: 
fcntl(/var/vmail/u...@example.com/.dovecot-sync.lock, write-lock, 
F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held 
by pid 628)
Nov 18 00:34:56 mx2 dovecot: 
dsync-local(u...@example.com)<9IKaB2KytF92AgAA5XpYKg>: Error: Couldn't 
lock /var/vmail/u...@example.com/.dovecot-sync.lock: 
fcntl(/var/vmail/u...@example.com/.dovecot-sync.lock, write-lock, 
F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held 
by pid 628)
Nov 18 00:39:26 mx2 dovecot: doveadm: Error: dsync(mx1.example.com): 
I/O has stalled, no activity for 600 seconds (last sent=mail_change 
(EOL), last recv=mailbox)
Nov 18 06:39:32 mx2 dovecot: doveadm: Error: dsync(mx1.example.com): 
I/O has stalled, no activity for 600 seconds (last sent=mail_change 
(EOL), last recv=mailbox)

*End*

I have configured "replication_full_sync_interval = 1 hours", which 
explains why some of the sync errors occur at the same increment on 
the hour (if the error does occur).


I've tested replication over tcps using either IPv6 or IPv4 -- this 
did not appear to make a difference.


Changing replication to occur over tcp solves the issue (with "ssl = 
yes" commented out, as well).


IMAP clients are primarily connecting to MX1 using SSL, which works 
well (SSL connections to MX2 also work). These are very low traffic 
machines at the moment (just 1 user as I continue testing).


I've attached the output of "dovecot -n" from each server.

Are there known bugs with replication using SSL? I'd appreciate any 
guidance.


Thank you,
AP

For what it's worth, I had the same issue when setting this up a few 
weeks ago. I switched to using SSH based transport and it's been great 
ever since. Is that an option for you?


dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u
mail_replica = remote:r...@xx.xx.xx.xx

Cheers
James




Re: Odd replication behaviour

2020-10-31 Thread James Pattinson
Solved. I knew this would happen. The act of writing it all out and including 
the configuration output gave me the solution.

I am using lmtp to deliver mail from postfix to Dovecot. I was missing the 
notify and replication plugins from 20-lmtp.conf

They were only present in 10-mail.conf as

mail_plugins = notify replication

Now, adding to 20-lmtp.conf:

protocol lmtp {
  mail_plugins = sieve notify replication
}

Works fine now. Hope this helps someone else.

Cheers
James

> On 31 Oct 2020, at 14:40, James Pattinson  wrote:
> 
> Hi,
> 
> I have just built a new pair of similar machines both running CentOS 8.2 
> (selinux disabled) and Dovecot 2.3.8 (9df20d2db).
> 
> One machine is a VPS (host A) and one is on my home network (host B). The 
> idea is that they are set up in a master/master config with Dovecot 
> replication.
> 
> I seem to have this 95% working but there is one strange issue I can’t work 
> out.
> 
> Currently B is a perfect replica of A. I have pointed an instance of 
> Thunderbird at it, and I can see all my mails. If I delete any mails or 
> change any flags, I see the same changes almost instantly on the A side.
> 
> PROBLEM: if host A receives a new mail, I don’t see it on B until I do 
> ‘something’ to change metadata, for example deleting any random email, or 
> marking an email as read on EITHER side causes the new email to appear almost 
> instantly on the B side.
> 
> I would have expected emails on B to appear immediately. Am I doing something 
> wrong?
> 
> Extra info -  my mailboxes are in Maildir format with single OS user (vmail). 
> I have about 4000 emails in the Inbox and about 30k in other folders.
> 
> There are only 5 users and I’m using passdb as the very simple backend.
> 
> Replication is via doveadm on a specified port (not SSH). Some output from 
> dovecot -n is below.
> 
> Cheers
> James
> 
> HOST A
> 
> # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.8 (b7b03ba2)
> # OS: Linux 4.18.0-193.28.1.el8_2.x86_64 x86_64 CentOS Linux release 8.2.2004 
> (Core)  xfs
> # Hostname: hosta.domain
> auth_mechanisms = plain login
> doveadm_password = # hidden, use -P to show it
> doveadm_port = 4040
> first_valid_uid = 1000
> mail_debug = yes
> mail_home = /srv/vmail/%u
> mail_location = maildir:/srv/vmail/%u
> mail_plugins = notify replication
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date index ihave 
> duplicate mime foreverypart extracttext
> mbox_write_locks = fcntl
> namespace inbox {
>  inbox = yes
>  location = 
>  mailbox Drafts {
>auto = subscribe
>special_use = \Drafts
>  }
>  mailbox Junk {
>auto = subscribe
>special_use = \Junk
>  }
>  mailbox Sent {
>special_use = \Sent
>  }
>  mailbox "Sent Messages" {
>special_use = \Sent
>  }
>  mailbox Trash {
>auto = subscribe
>special_use = \Trash
>  }
>  prefix = 
> }
> passdb {
>  args = scheme=BLF-CRYPT username_format=%u /etc/dovecot/users
>  driver = passwd-file
> }
> plugin {
>  mail_replica = tcp:b.b.b.b:4040
>  sieve = file:~/sieve;active=~/.dovecot.sieve
>  sieve_before = /var/mail/SpamToJunk.sieve
> }
> protocols = imap lmtp
> service aggregator {
>  fifo_listener replication-notify-fifo {
>group = root
>mode = 0660
>user = vmail
>  }
>  unix_listener replication-notify {
>group = root
>mode = 0660
>user = vmail
>  }
> }
> service auth {
>  unix_listener /var/spool/postfix/private/auth {
>group = postfix
>mode = 0600
>user = postfix
>  }
> }
> service doveadm {
>  inet_listener {
>port = 4040
>  }
> }
> service lmtp {
>  unix_listener /var/spool/postfix/private/dovecot-lmtp {
>group = postfix
>mode = 0600
>user = postfix
>  }
> }
> service replicator {
>  process_min_avail = 1
>  unix_listener replicator-doveadm {
>mode = 0600
>user = vmail
>  }
> }
> ssl = required
> ssl_cert =  ssl_cipher_list = PROFILE=SYSTEM
> ssl_dh = # hidden, use -P to show it
> ssl_key = # hidden, use -P to show it
> ssl_min_protocol = TLSv1.2
> ssl_prefer_server_ciphers = yes
> userdb {
>  args = username_format=%u /etc/dovecot/users
>  default_fields = uid=vmail gid=mail home=/srv/vmail/%u
>  driver = passwd-file
> }
> protocol lmtp {
>  mail_plugins = sieve
> }
> protocol lda {
>  mail_plugins = notify replication sieve
> }
> 
> HOST B
>

Odd replication behaviour

2020-10-31 Thread James Pattinson
Hi,

I have just built a new pair of similar machines both running CentOS 8.2 
(selinux disabled) and Dovecot 2.3.8 (9df20d2db).

One machine is a VPS (host A) and one is on my home network (host B). The idea 
is that they are set up in a master/master config with Dovecot replication.

I seem to have this 95% working but there is one strange issue I can’t work out.

Currently B is a perfect replica of A. I have pointed an instance of 
Thunderbird at it, and I can see all my mails. If I delete any mails or change 
any flags, I see the same changes almost instantly on the A side.

PROBLEM: if host A receives a new mail, I don’t see it on B until I do 
‘something’ to change metadata, for example deleting any random email, or 
marking an email as read on EITHER side causes the new email to appear almost 
instantly on the B side.

I would have expected emails on B to appear immediately. Am I doing something 
wrong?

Extra info -  my mailboxes are in Maildir format with single OS user (vmail). I 
have about 4000 emails in the Inbox and about 30k in other folders.

There are only 5 users and I’m using passdb as the very simple backend.

Replication is via doveadm on a specified port (not SSH). Some output from 
dovecot -n is below.

Cheers
James

HOST A

# 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.8 (b7b03ba2)
# OS: Linux 4.18.0-193.28.1.el8_2.x86_64 x86_64 CentOS Linux release 8.2.2004 
(Core)  xfs
# Hostname: hosta.domain
auth_mechanisms = plain login
doveadm_password = # hidden, use -P to show it
doveadm_port = 4040
first_valid_uid = 1000
mail_debug = yes
mail_home = /srv/vmail/%u
mail_location = maildir:/srv/vmail/%u
mail_plugins = notify replication
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext
mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
auto = subscribe
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix = 
}
passdb {
  args = scheme=BLF-CRYPT username_format=%u /etc/dovecot/users
  driver = passwd-file
}
plugin {
  mail_replica = tcp:b.b.b.b:4040
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_before = /var/mail/SpamToJunk.sieve
}
protocols = imap lmtp
service aggregator {
  fifo_listener replication-notify-fifo {
group = root
mode = 0660
user = vmail
  }
  unix_listener replication-notify {
group = root
mode = 0660
user = vmail
  }
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0600
user = postfix
  }
}
service doveadm {
  inet_listener {
port = 4040
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
mode = 0600
user = vmail
  }
}
ssl = required
ssl_cert = 

Re: iterate_query with static userdb ?

2020-05-17 Thread James

On 17/05/2020 10:43, Laura Smith wrote:


Because I wanted to avoid storing uid/gid/home in the database ?


I use:

user_query = "SELECT 'vmail' AS uid, 'vmail' AS gid, allow_nets, 
'*:storage=' || quota || 'M' AS quota_rule FROM mailbox WHERE username = 
'%n' AND domain = '%d';"


... uid and gid are not stored in my database but are returned by the query.


Re: dovecot sieve duplicates detection

2019-12-04 Thread James Cassell via dovecot


On Wed, Dec 4, 2019, at 1:14 PM, Stephan Bosch via dovecot wrote:
> 
> 
> On 17/08/2018 09:14, Stephan Bosch wrote:
> >
> >
> > Op 14/05/2018 om 23:03 schreef James Cassell:
> >>
> >> On Mon, May 14, 2018, at 4:52 PM, Stephan Bosch wrote:
> >>>
> >>> Op 25/04/2018 om 22:49 schreef James Cassell:
> >>>> On Wed, Apr 25, 2018, at 3:20 PM, Stephan Bosch wrote:
> >>>>> Specify the ID used for duplicate checking explicitly using the
> >>>>> :uniqueid argument (https://tools.ietf.org/html/rfc7352#section-3.1).
> >>>>> Using the variables extenion, compose the uniqueid from the 
> >>>>> message-id
> >>>>> and the mailbox name.
> >>>>>
> >>>> In my experience with dovecot's implementation, you can set the ID 
> >>>> only once in a script.  If you try to filter duplicates based on 
> >>>> multiple IDs, only the first (or last, I don't remember) takes effect.
> >>>>
> >>> Do you have a detailed example of the supposed wrong behavior?
> >>>
> >> I don't have them readily available. Basically, the result of the 
> >> first duplicate test in a script is taken as the result of any future 
> >> duplicate test, even if the parameters to that future duplicate test 
> >> in the same script are different and would otherwise result in a 
> >> different output. The duplicate test is only evaluated once and its 
> >> results are substituted everywhere.
> >>
> >> For example, I might want to flag a message as a new conversation if 
> >> I have not seen another message with the same subject. In the same 
> >> script, I might want to discard messages that are exactly identical 
> >> including message ID among others. The dovecot behavior would be to 
> >> discard all messages that match a subject of previously received 
> >> message.
> >
> > I finally managed to review this issue and I can confirm that this is 
> > a bug.
> 
> Fix released in 2.3.9.
> 

Awesome! Thanks for the followup!

V/r,
James Cassell


Re: Duplicate e-mail with Dovecot and Sieve

2019-11-26 Thread James Cassell via dovecot
On Tue, Nov 26, 2019, at 5:13 AM, Claudio Corvino via dovecot wrote:
>  
> Hi everyone,
> 
> we have a duplicate e-mail problem with Dovecot 2.2.13 (LMTP as MDA) on 
> a Debian 8 server.
> 
> Our users are on external LDAP.
> 
> We just need to setup a vacation responder (with Sieve) for our 
> employees with a redirect to an alias (a distribution list), let's call 
> it *l...@domain.com*, that comprehends even the user that just 
> activated the out of office.
> 
> This generates a loop: when someone write to l...@domain.com every user 
> of the aforementioned distribution list receives a duplicate e-mail 
> generated from the sieve script of the user with vacation responder 
> active that contains the redirect to l...@domain.com.
> 
> We just tried activating the duplicate extension into sieve script but 
> the redirected e-mail has a different MSG-ID and it's not recognized as 
> a duplicate by the server.
> 
> Our dovecot conf is:
> 
[snip]
> An example of user sieve script is:
> 
> *require ["vnd.dovecot.duplicate","copy","fileinto","vacation-seconds"];
>  # rule:[Outofoffice]
>  if true
>  {
>  vacation :addresses "...@domain.tld" :subject "Automatic Reply" text:
> *
> 
> *
>  .
>  ;
>  fileinto "INBOX";*
> 
> * redirect "l...@domain.com";*
> 
> *}*
> 

You've not used the duplicate test anywhere in your example.  See the RFC for 
examples:
https://tools.ietf.org/html/rfc7352#section-5


Specifically, example 3 might help: 
https://tools.ietf.org/html/rfc7352#section-5.3

but most simply, you can probably replace `if true` with `if not duplicate`


V/r,
James Cassell


Re: SQL iterate_query

2019-10-28 Thread James via dovecot

On 25/10/2019 00:00, Daniel Miller via dovecot wrote:


So, given that the complete address is used as the username I now use:
iterate_query = SELECT username FROM mailbox


https://doc.dovecot.org/configuration_manual/authentication/sql/#user-iteration

...says that iterate_query has two variants.  You have complete 
addresses in a database column "username" so you need the first.  The 
addresses must be return in a column named "user", hence add "AS user" 
to label your column.  Try:


iterate_query = SELECT username AS user FROM mailbox







Re: Error: SSL_accept() syscall failed

2019-10-11 Thread C. James Ervin via dovecot
bit ECDH (P-384)
 OpenSSL 1.0.2e   TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384, 384 
bit ECDH (P-384)
 OpenSSL 1.1.0j (Debian)  TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384, 384 
bit ECDH (P-384)
 OpenSSL 1.1.1b (Debian)  TLSv1.3 TLS_AES_256_GCM_SHA384, 384 bit 
ECDH (P-384)
 Thunderbird (60.6)   TLSv1.3 TLS_AES_128_GCM_SHA256, 384 bit 
ECDH (P-384)


 Done 2019-10-11 07:31:08 [ 170s] -->> 3.222.54.62:993 (kumo.kites.org) 
<<--



On 10/11/19 7:22 AM, C. James Ervin via dovecot wrote:


In setting up my new mail server, I am getting the following in the logs:

Oct 11 07:10:59 kumo dovecot[5704]: imap-login: Disconnected (no auth 
attempts in 0 secs): user=<>, rip=24.53.79.10, lip=172.26.12.90, *TLS 
handshaking: SSL_accept() syscall failed: Success*, 
session=





Error: SSL_accept() syscall failed

2019-10-11 Thread C. James Ervin via dovecot

In setting up my new mail server, I am getting the following in the logs:

Oct 11 07:10:59 kumo dovecot[5704]: imap-login: Disconnected (no auth 
attempts in 0 secs): user=<>, rip=24.53.79.10, lip=172.26.12.90, *TLS 
handshaking: SSL_accept() syscall failed: Success*, 
session=


I have tried various ssl_protocols entries, but for now have defaulted 
back to ssl_protocols = !SSLv3 (the "out of the box" setting).


The certificate (ssl_cert = valid and the same one used by postfix.  However, I belive the error 
appeared only after removing the self signed certificate and installing 
the one I purchased.


Warning:  though I was a sysadmin a long long time, it has been a 
while!  I have not run a mail server in 15 years, but am moving my mail 
back "in-house" as I approach retirement!  I searched for a forum where 
I could post this issue, but only found this list.




# 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf

# Pigeonhole version 0.4.21 (92477967)
# OS: Linux 4.15.0-1051-aws x86_64 Ubuntu 18.04.1 LTS
auth_mechanisms = plain login
mail_location = maildir:~/Maildir
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart extracttext

namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
}
passdb {
  driver = pam
}
plugin {
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
}
protocols = " imap sieve pop3"
service auth {
  unix_listener /var/spool/postfix/private/dovecot-auth {
    group = postfix
    mode = 0660
    user = postfix
  }
}
ssl_cert =  was automatically rejected:%n%r
}
protocol imap {
  imap_client_workarounds = delay-newmail
  mail_max_userip_connections = 10
}
protocol pop3 {
  mail_max_userip_connections = 10
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
}


Re: Case sensitivity in :addresses in sieve vacation scripts

2019-10-08 Thread James Cassell via dovecot
On Tue, Oct 8, 2019, at 7:58 AM, Julian Kippels via dovecot wrote:
> Hi,
> 
> I have recently updated from Dovecot 2.2 to 2.3. Since I have noticed
> that vacation responses from sieve are not working the same anymore.
> For example, my sieve script looks like this:
> 
>  vacation :days 1 :addresses ["kipp...@hhu.de","julian.kipp...@hhu.de"]
> 
> it used to be that I got a vacation response if I sent a mail to
> kipp...@hhu.de and kipp...@hhu.de. Now I only get a response for
> kipp...@hhu.de, not for kipp...@hhu.de.
> I cant say for sure, but I suppose this behaviour changed with the
> update. If not, why could this have happened? And in any case, how can
> I enable case insensitivity?
> 

I took a look at the RFCs.  It appears that this change is not in accordance 
with the relevant standards.  SIEVE says 'the "i;ascii-casemap" comparator 
(which treats uppercase and lowercase characters in the US-ASCII subset of 
UTF-8 as the same).  If left unspecified, the default is "i;ascii-casemap".' 
https://tools.ietf.org/html/rfc5228#section-2.7.3

Since there is no mention of comparators in the Vacation RFC, it should 
fallback to case-insensitive: https://tools.ietf.org/html/rfc5230

You could try working around the issue by adding to your `vacation` statement: 
`:comparator "i;ascii-casemap"` -- but in any case, I'd file a bug about the 
non-standard behavior.


V/r,
James Cassell


> Thanks
> Julian
> 
>


Off-site cloud backup (eg Amazon S3, Wasabi)

2019-09-09 Thread James Brown via dovecot
I’d like to synchronise our mail store to an Amazon S3 bucket or similar, for 
protection from ransomware.

Server is running on macOS with maildir.

Should I use a Dovecot process, rsync, a Mac app like Arq or Jungle Disk, or 
something else?

Anyone have experience or opinions on this?

Thanks,

James.




smime.p7s
Description: S/MIME cryptographic signature


Solr, Dovecot & macOS / iOS

2019-08-13 Thread James Brown via dovecot
I’m thinking of getting Solr working with my Dovecot server. Server is new 
6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of 
messages.

But I’m not sure if:

a) it will make enough of a difference and

b) does Mail.app and other mail clients on Macs or iOS devices perform searches 
on their local copy of mail or does it just send a search request to the server?

I’m guessing the searches are done locally so no point in Solr?

Does anyone know? 

Thanks, James.


smime.p7s
Description: S/MIME cryptographic signature


Re: auth-policy crashing

2019-08-07 Thread James via dovecot

On 07/08/2019 11:19, James via dovecot wrote:


My more simplistic policy does not need both.  I perform whitelist,
blacklist, geo and greylist


...and DNSBL which where I started with the policyserver, "Can dovecot 
do DNSBL?", only indirectly via a policyserver.  This is better as most 
pass white list or fail geo local checks before doing the external DNS 
lookup.


Re: auth-policy crashing

2019-08-07 Thread James via dovecot

On 07/08/2019 11:02, Aki Tuomi via dovecot wrote:


before and after auth?  roundcube webmail reports an error with only
auth_policy_check_before_auth.  I cannot see why.  The simple and lazy
solution is to use double auth_policy_check_!

...


The double-check is for places which want to implement something like
COS or want to perform validations in policy server *after* we know the
user identity. The first check is done before we even know if the user
or the credential(s) are valid.


I can see why both before and after are options.  My more simplistic 
policy does not need both.  I perform whitelist, blacklist, geo and 
greylist and do not cross reference these with the user.  I can't see 
why roundcubemail fails without both.  The IMAP exchange with 
roundcubemail should not be aware of the policy server.  I was spending 
[wasting] too much time on looking for an answer and gave up.


Re: auth-policy crashing

2019-08-07 Thread James via dovecot

On 06/08/2019 06:46, Aki Tuomi via dovecot wrote:


On 2.8.2019 13.45, James via dovecot wrote:

My auth process is dumping core.  This happens several times per day

...


There is an easy fix for this, attached.


Patch applied; no core dump in 24 hours.

This appears to have fixed the problem.  I found that it crashed when 
the policy server responded too quickly.  As the before and after auth 
command=allow request are the same I cache the first, leading to a fast 
second response.  Removing the cache (nginx proxy_cache ...) must change 
the timings and circumvented the crash.  Why use both check before and 
after auth?  roundcube webmail reports an error with only 
auth_policy_check_before_auth.  I cannot see why.  The simple and lazy 
solution is to use double auth_policy_check_!


Thank you Aki for looking at this and finding a solution so quickly.


Re: auth-policy crashing

2019-08-03 Thread James via dovecot

On 02/08/2019 11:45, James via dovecot wrote:

My auth process is dumping core.  This happens several times per day but
dovecot can operate normally for hours between errors.

The crash occurs in src/auth/auth-policy.c, line 356:

t@1 (l@1) program terminated by signal SEGV (no mapping at the fault
address)
Current function is auth_policy_parse_response
  356   context->request->policy_refusal = FALSE;



Further tracking shows this sets context->request to NULL:

"src/lib/iostream.c" line 54

array_foreach(>destroy_callbacks, dc)
dc->callback(dc->context);


Very occasionally I see:

Aug  3 11:00:35 mailhost dovecot: [ID 702911 mail.crit] auth: Panic: 
file http-client-request.c: line 283 (http_client_request_unref): 
assertion failed: (req->refcount > 0)



Swapping keep-alive on/off changes crashing from very approximately once 
per day to some per hour.  I guess there is some fundamental thread 
clash or keep alive time out clean-up failure.




James.




auth-policy crashing

2019-08-02 Thread James via dovecot
My auth process is dumping core.  This happens several times per day but 
dovecot can operate normally for hours between errors.


The crash occurs in src/auth/auth-policy.c, line 356:

t@1 (l@1) program terminated by signal SEGV (no mapping at the fault 
address)

Current function is auth_policy_parse_response
  356   context->request->policy_refusal = FALSE;

...context->request is null.  Add markers to the code...

}
if (context->request == NULL) fprintf(stderr, 
" context->request == NULL\n");

i_stream_unref(>payload);

if (context->request == NULL) fprintf(stderr, 
" context->request == NULL\n");

if (context->parse_error) {
context->result = (context->set->policy_reject_on_fail ? -1 : 0);
}

if (context->request == NULL) fprintf(stderr, 
" context->request == NULL\n");

context->request->policy_refusal = FALSE;

if (context->result < 0) {

...gives at the time of the crash...

Aug  1 14:25:44 mailhost dovecot: [ID 702911 mail.error] auth: Error: 
 context->request == NULL
Aug  1 14:25:44 mailhost dovecot: [ID 702911 mail.error] auth: Error: 
 context->request == NULL


...so context->result is not null before the call (no 222) to 
i_stream_unref but is after.




dovecot.conf has:

auth_policy_server_url = http://policyserver.lan/
auth_policy_server_timeout_msecs = 3000
auth_policy_hash_nonce = Ohr9phaeSeip2Pahaez2raiGohxoo5Ia
auth_policy_request_attributes = remote=%{rip}
auth_policy_check_before_auth = yes
auth_policy_check_after_auth = yes
auth_policy_report_after_auth = yes


To simplify the problem I used a dummy policy server, in nginx.conf:

location / {
default_type  application/json;
return 200 "{\"status\":0,\"msg\":\"accepted\"}";
}

however no matter what rubbish a policy server sends back it should not 
cause dovecot to crash.


I've tried 32 and 64 bit and two compilers (gcc and SunStudio), all 
result in crashes.  Adding:

keepalive_timeout 0;
...to nginx.conf appears to reduce the crashes.  It happens with a 
variety of users and with debug output I see no pattern.




James.
















Re: Dovecot 2.3.6 on Solaris10: build issues, segfaults

2019-07-10 Thread James via dovecot

On 09/07/2019 06:35, Aki Tuomi via dovecot wrote:


 As for 1st, isn't gcc 3 rather old?


As is Solaris 10.  Solaris 10 predates gcc4 and comes/came with gcc3. No 
one says we have keep using the system gcc3 exclusively. 9.1.0 works too.


Solaris 10 03/05 = March 2005, release January 31, 2005 [1]
gcc 4.0.0: April 20, 2005 [2]


$ pkginfo -l SUNWgcc
   PKGINST:  SUNWgcc
  NAME:  gcc - The GNU C compiler
  CATEGORY:  system
  ARCH:  i386
   VERSION:  11.10.0,REV=2005.01.08.01.09
   BASEDIR:  /
VENDOR:  Oracle Corporation
  DESC:  GNU C - The GNU C compiler 3.4.3
PSTAMP:  sfw10-patch-x20121120060015
...



1. 
https://en.wikipedia.org/wiki/Solaris_%28operating_system%29#Version_history

"supported until 2021"

2. https://www.gnu.org/software/gcc/releases.html


Re: Dovecot 2.3.6 on Solaris10: build issues, segfaults

2019-07-10 Thread James via dovecot

On 09/07/2019 01:02, Joseph Tam via dovecot wrote:


Issue 2) Cannot build with --enable-hardening

Using gcc 9.1.0, "configure" step fails because fd passing was
broken, but the real problem was a compilation failure when
"--enable-hardening" is used.  Demonstration:


See:
https://dovecot.org/pipermail/dovecot/2019-January/114121.html



Issue 3) dovecot/doveconf segfaults on startup

It crashes here while processing dovecot.conf, as does "doveconf"

(settings-parser.c:1519 in setting_copy())
*dest_size = *src_size;

It appears *src_size is not an 8-byte address aligned (0x5597c).
It inherits this value from the calling routine as the sum of
"set" (8-byte aligned) + "def->offset"=20 => misaligned address.


32 bit or 64bit? cflags? I use 32 bit cc 12.6 and have no problem.  My 
test with gcc 9.1.0 didn't dump core either.


Re: Compile problem on Solaris

2019-06-10 Thread James via dovecot

On 10/06/2019 14:40, Roderick Johnstone via dovecot wrote:

Compiling dovecot 2.2.36.1 with the native compiler on Solaris 10 is 
giving the error below.

...

-m64 -c -o sha3.lo sha3.c
libtool: compile:  cc -DHAVE_CONFIG_H -I. -I../.. 
-I/export/home/exim_build_20190605/build_local/include -m64 -c sha3.c 
-KPIC -DPIC -o .libs/sha3.o

"sha3.h", line 49: warning: unnamed struct member

...

Untested: try naming the union. My own builds (dovecot 2.3.6) get 
through this.  What compiler version? 12.6 is current and does not issue 
the unnamed struct warning, eg, 12.3 does.


struct sha3_ctx {
uint64_t saved;  /* the portion of the input message that we
  * didn't consume yet */
union {  /* Keccak's state */
uint64_t s[SHA3_KECCAK_SPONGE_WORDS];
uint8_t sb[SHA3_KECCAK_SPONGE_WORDS * 8];
} X;



Re: v2.3.6 Configure fail 'libssl not found' macOS

2019-05-01 Thread James Brown via dovecot
> On 1 May 2019, at 4:38 pm, James Brown via dovecot  <mailto:dovecot@dovecot.org>> wrote:
> 
> No problems with 2.3.5, but when I try to compile 2.3.6 with:
> 
> ./configure --with-mysql --with-ssl=openssl
> 
> I get:
> 
> checking for dlopen... yes
> checking for SSL_read in -lssl... no
> configure: error: Can't build with OpenSSL: libssl not found
> 
> $ locate libssl
> /Applications/BBEdit.app/Contents/XPCServices/GitIgnoreXPCService.xpc/Contents/Frameworks/libssl.dylib
> /Applications/Navicat for MySQL.app/Contents/Frameworks/libssl.1.0.0.dylib
> /Applications/assp/assp.mod/bin/libssl32.dll
> /usr/lib/libssl.0.9.7.dylib
> /usr/lib/libssl.0.9.8.dylib
> /usr/lib/libssl.35.dylib
> /usr/lib/libssl.43.dylib
> /usr/lib/libssl.44.dylib
> /usr/lib/libssl.dylib
> /usr/local/Cellar/openssl/1.0.2r/lib/libssl.1.0.0.dylib
> /usr/local/Cellar/openssl/1.0.2r/lib/libssl.a
> /usr/local/Cellar/openssl/1.0.2r/lib/libssl.dylib
> /usr/local/Cellar/openssl/1.0.2r/lib/pkgconfig/libssl.pc
> /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.1.1.dylib
> /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.a
> /usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.dylib
> /usr/local/Cellar/openssl@1.1/1.1.1b/lib/pkgconfig/libssl.pc
> /usr/local/lib/dovecot/libssl_iostream_openssl.a
> /usr/local/lib/dovecot/libssl_iostream_openssl.la 
> <http://libssl_iostream_openssl.la/>
> /usr/local/lib/dovecot/libssl_iostream_openssl.so
> 
> Any suggestions?
> 
> Thanks,
> 
> James.

Sorry for the noise.

export CPPFLAGS="-I/usr/local/opt/mysql@5.7/include/mysql 
-I/usr/local/Cellar/openssl@1.1/1.1.1b/include"
export LDFLAGS="-L/usr/local/opt/mysql@5.7/lib -L/usr/local/opt/openssl@1.1/lib"

./configure --with-mysql --with-ssl=openssl

Worked.

James.

smime.p7s
Description: S/MIME cryptographic signature


v2.3.6 Configure fail 'libssl not found' macOS

2019-05-01 Thread James Brown via dovecot
No problems with 2.3.5, but when I try to compile 2.3.6 with:

./configure --with-mysql --with-ssl=openssl

I get:

checking for dlopen... yes
checking for SSL_read in -lssl... no
configure: error: Can't build with OpenSSL: libssl not found

$ locate libssl
/Applications/BBEdit.app/Contents/XPCServices/GitIgnoreXPCService.xpc/Contents/Frameworks/libssl.dylib
/Applications/Navicat for MySQL.app/Contents/Frameworks/libssl.1.0.0.dylib
/Applications/assp/assp.mod/bin/libssl32.dll
/usr/lib/libssl.0.9.7.dylib
/usr/lib/libssl.0.9.8.dylib
/usr/lib/libssl.35.dylib
/usr/lib/libssl.43.dylib
/usr/lib/libssl.44.dylib
/usr/lib/libssl.dylib
/usr/local/Cellar/openssl/1.0.2r/lib/libssl.1.0.0.dylib
/usr/local/Cellar/openssl/1.0.2r/lib/libssl.a
/usr/local/Cellar/openssl/1.0.2r/lib/libssl.dylib
/usr/local/Cellar/openssl/1.0.2r/lib/pkgconfig/libssl.pc
/usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.1.1.dylib
/usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.a
/usr/local/Cellar/openssl@1.1/1.1.1b/lib/libssl.dylib
/usr/local/Cellar/openssl@1.1/1.1.1b/lib/pkgconfig/libssl.pc
/usr/local/lib/dovecot/libssl_iostream_openssl.a
/usr/local/lib/dovecot/libssl_iostream_openssl.la 
<http://libssl_iostream_openssl.la/>
/usr/local/lib/dovecot/libssl_iostream_openssl.so

Any suggestions?

Thanks,

James.

smime.p7s
Description: S/MIME cryptographic signature


Re: Dovecot release v2.3.6

2019-04-30 Thread James via dovecot

On 30/04/2019 14:21, Aki Tuomi via dovecot wrote:


https://dovecot.org/releases/2.3/dovecot-2.3.6.tar.gz


Trivial but...

"mail-index-transaction-update.c", line 198: void function cannot return 
value



Thanks.
--- ../original/src/lib-index/mail-index-transaction-update.c   2019-04-30 
13:25:06.0 +
+++ src/lib-index/mail-index-transaction-update.c   2019-04-30 
14:49:09.517684762 +
@@ -195,7 +195,8 @@
   uint32_t first_uid,
   ARRAY_TYPE(seq_range) *uids_r)
 {
-   return mail_index_append_finish_uids_full(t, first_uid, first_uid, 
uids_r);
+   mail_index_append_finish_uids_full(t, first_uid, first_uid, uids_r);
+   return;
 }
 
 void mail_index_append_finish_uids_full(struct mail_index_transaction *t,


Re: Mail account brute force / harassment

2019-04-12 Thread James via dovecot

On 12/04/2019 08:42, Aki Tuomi via dovecot wrote:

On 12.4.2019 10.34, James via dovecot wrote:

On 12/04/2019 08:24, Aki Tuomi via dovecot wrote:


Weakforced uses Lua so you can easily integrate DNSBL support into it.

How does this help Dovecot block?
A link to some documentation or example perhaps?



https://wiki.dovecot.org/Authentication/Policy

You can configure weakforced to return status -1 when DNSBL matches,
which causes the user authentication to fail before any other processing
happens.


Thank you.  I will study this - although I dispute your "easily"!



James.



Re: Mail account brute force / harassment

2019-04-12 Thread James via dovecot

On 12/04/2019 08:24, Aki Tuomi via dovecot wrote:


Weakforced uses Lua so you can easily integrate DNSBL support into it.


How does this help Dovecot block?
A link to some documentation or example perhaps?



We will not add DNSBL support to dovecot at this time.


Is there a reason why you will not support this RFE?



Re: Mail account brute force / harassment

2019-04-12 Thread James via dovecot

On 11/04/2019 14:33, Anton Dollmaier via dovecot wrote:


Which is why a dnsbl for dovecot is a good idea.  I do not believe the
agents behind these login attempts are only targeting me, hence the
addresses should be shared via a dnsbl.


Probably there's an existing solution for both problems (subsequent
attempts and dnsbl):


https://github.com/PowerDNS/weakforced


"The goal of 'wforce' is to detect brute forcing of passwords across 
many servers"


The problem is not detecting but blocking.  Dovecot has no mechanism for 
using the data; Dovecot needs DNSBL capability.


I tested a small sample of my IMAP hackers using the lists I use for 
SMTP blocking [1] and enough are in these list to make them worth using. 
 Extra detection is not needed as many of these addresses are already 
known - maybe even by using weakforced.




James.


1. exim dnsblist:
https://www.exim.org/howto/rbl.html
https://www.exim.org/exim-html-current/doc/html/spec_html/ch-access_control_lists.html



Re: Mail account brute force / harassment

2019-04-11 Thread James via dovecot

On 11/04/2019 12:49, Marc Roos via dovecot wrote:

Yes indeed, we have already own dnsbl's for smtp and ssh/ftp access. How
do you have one setup for dovecot connections?


Two answers:

1. I wrote my own very simple implementation but it does not share other 
people's data.  Sharing the key to viability so it is/was a pointless 
exercise.  Without sharing a hacker gets at least one free shot per 
server per address.  With sharing it is closer to one per address and 
less with honeypots.



2. I said "dnsbl for dovecot is a good idea", an idea.  When this was 
raised previously we were told it was not needed and it can all be done 
with tcp wrappers, fail2ban and allow_nets.


https://dovecot.org/list/dovecot/2013-July/091236.html
https://dovecot.org/list/dovecot/2014-June/096662.html



Re: Mail account brute force / harassment

2019-04-11 Thread James via dovecot

On 11/04/2019 11:43, Marc Roos via dovecot wrote:


A. With the fail2ban solution
   - you 'solve' that the current ip is not able to access you


It is only a solution if there are subsequent attempts from the same 
address.  I currently have several thousand addresses blocked due to 
dovecot login failures.  My firewall is set to log these so I can see 
that few repeat, those that do repeat have intervals of >1 week. 
Blocking these has minimal effect (other than to clog fail12ban and the 
firewall).



   - it will continue bothering other servers and admins


Which is why a dnsbl for dovecot is a good idea.  I do not believe the 
agents behind these login attempts are only targeting me, hence the 
addresses should be shared via a dnsbl.





Re: sql table definitions

2019-04-02 Thread James via dovecot

On 02/04/2019 05:42, Richard Hector via dovecot wrote:


I'm using PostgreSQL for my auth db. I used the example CREATE TABLE
statement in the config file, but now I find the fields are too short. I
assume dovecot will be fine with 'text' type columns replacing the
varchars? Or failing that, I can change the length of the varchar fields?


ALTER TABLE $table ALTER COLUMN $column TYPE VARCHAR($newlenth);


My tables are very different from the dovecot suggested tables and it 
works, just make sure the queries in dovecot-sql.conf correspond.  My 
tables are different because they hold additional information for 
routing (the db is shared with exim).







dovecot.conf "local hostname" uses only one resolved value

2019-03-14 Thread James via dovecot

https://wiki.dovecot.org/SSL/DovecotConfiguration#Different_certificates_per_IP_and_protocol

says:
local 192.0.2.10 { # instead of IP you can also use hostname, which will 
be resolved


However if the name resolves to multiple values only one is used.

Test.
Choose any name with multiple values, I created a local name:

$ nslookup multi.lan
Server: 127.0.0.1
Address:127.0.0.1#53

Name:   multi.lan
Address: 192.168.1.2
Name:   multi.lan
Address: 192.168.1.3
Name:   multi.lan
Address: 192.168.1.1


Minimal dovecot.conf:

local multi.lan {
  protocol imap {
ssl_cert = In my real case with A and  records, only the  record is used. 
Testing with "openssl s_client -starttls imap ..."  gives me the wrong 
certificate for the IPv4 address.  Workaround: specify all addresses and 
do not use lookup.




James.




“Message cannot be displayed because of the way it is formatted” iOS

2019-03-06 Thread James Brown via dovecot
Have updated our mail server and are now running Dovecot 2.3.4. Things are 
mostly working OK, but occasionally I have users on latest iOS getting this 
message: “this message cannot be displayed because of the way it is formatted. 
Ask the sender to send it again using a different format or email program. 
multipart/alternative”

The email seems to open OK on the macOS version of Mail.app.

Also getting “Loading...” displayed in body section of the app with content 
never displayed. 

Most emails are fine, but just get this strange ones occasionally. Our previous 
mail server was much older, with older version of Dovecot, etc. This server 
never used to have these issues.

Just wondering if anyone has any ideas or suggestions? Can’t see anything in 
the logs.

Thanks, James.



smime.p7s
Description: S/MIME cryptographic signature


same account imap and pop3

2019-03-01 Thread James via dovecot

I have an imap and pop3 for the same account.
When I delete a message from imap and then fetch the mail from the pop3 
account, it retrieves the deleted message.

I am using thunderbird for the MTA.
Does anyone know why it happens?
It could be a timing issue.



Re: "unknown user - trying the next userdb" Info in log

2019-01-29 Thread James Brown via dovecot
On 30 Jan 2019, at 4:35 pm, Aki Tuomi mailto:aki.tu...@open-xchange.com>> wrote:
> 
>> 
>> On 30 January 2019 at 07:12 James Brown < jlbr...@bordo.com.au 
>> <mailto:jlbr...@bordo.com.au>> wrote:
>> 
>> 
>> >> My settings:
>> ...
>> >> userdb {
>> >> driver = passwd
>> >> }
>> >> userdb {
>> >> driver = prefetch
>> >> }
>> >> userdb {
>> >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
>> >> driver = sql
>> >> }
>>> Well... there is that usetdb passwd which seems bit extraneous.
>>> ---
>>> Aki Tuomi
>> 
> I'd remove the
> 
> userdb {
> driver = passwd
> }
> 
> section
> --- 
> Aki Tuomi

Thanks Aki - the trick was finding where that setting was! Found it in 
auth-system.conf.ext.

Commented it out and all works perfectly now.

Thanks again Aki,

James.

Re: "unknown user - trying the next userdb" Info in log

2019-01-29 Thread James Brown via dovecot
On 30 Jan 2019, at 10:57 am, Stephan Bosch mailto:step...@rename-it.nl>> wrote:
> 
> Op 30/01/2019 om 00:06 schreef James Brown via dovecot:
>> 
>>> On 30 Jan 2019, at 9:24 am, Stephan Bosch >> <mailto:step...@rename-it.nl>> wrote:
>>> 
>>> 
>>> 
>>> Op 29/01/2019 om 01:51 schreef James Brown via dovecot:
>>>> I’ve set up a new mail server. Auth-worker is giving me this error all the 
>>>> time:
>>>> 
>>>> auth-worker(11160): Info: passwd(sa...@bordo.com.au 
>>>> <mailto:sa...@bordo.com.au> <mailto:sa...@bordo.com.au 
>>>> <mailto:sa...@bordo.com.au>>,10.0.0.54,): unknown user  
>>>> - trying the next userdb
>>>> 
>>> That is not an error. As the "Info: " prefix indicates, this is an 
>>> innocuous informational message.
>>> 
>>> Regards,
>>> 
>>> Stephan.
>>> 
>> Thanks Stephan.
>> 
>> I know it’s not a major thing, it’s just that it is every third line of my 
>> log.
>> 
>> Ie I get the auth-worker unknown user, then successful imap-login, then the 
>> imap logout.
>> 
>> So I’m just trying to figure out what is causing the auth-worker lookup 
>> failure so I can fix it. I did not have this in my old mail server, just the 
>> new one I’ve setup.
>> 
>> Is it a database issue or a Dovecot config issue? If so where?
> 
> Ýou can enable auth_verbose and auth_debug to get an idea what Dovecot is 
> doing. You should post your `dovecot -n` to get more help on this issue.
> 
> Regards,
> 
> Stephan.

Will look at auth_debug. It is a lot to go through!

My settings:

$ sudo dovecot -n
Password:
# 2.3.4 (0ecbaf23d): /usr/local/etc/dovecot/dovecot.conf
# OS: Darwin 18.2.0 x86_64  apfs
# Hostname: mail.bordo.com.au <http://mail.bordo.com.au/>
auth_debug = yes
auth_verbose = yes
auth_verbose_passwords = plain
default_internal_user = jlbrown
default_login_user = jlbrown
first_valid_gid = 0
first_valid_uid = 102
last_valid_gid = 501
last_valid_uid = 105
log_path = /var/log/dovecot.log
mail_gid = 102
mail_location = maildir:/usr/local/virtual/%d/%u
mail_uid = 102
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix = 
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
protocols = imap pop3
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = _postfix
mode = 0660
user = _postfix
  }
  unix_listener auth-userdb {
mode = 0660
user = _vmail
  }
}
service imap-login {
  process_limit = 512
}
service stats {
  fifo_listener stats-mail {
mode = 0666
  }
  fifo_listener stats-user {
mode = 0666
  }
  unix_listener stats-writer {
mode = 0666
  }
}
ssl_cert = 

Re: "unknown user - trying the next userdb" Info in log

2019-01-29 Thread James Brown via dovecot



> On 30 Jan 2019, at 9:24 am, Stephan Bosch  wrote:
> 
> 
> 
> Op 29/01/2019 om 01:51 schreef James Brown via dovecot:
>> I’ve set up a new mail server. Auth-worker is giving me this error all the 
>> time:
>> 
>> auth-worker(11160): Info: passwd(sa...@bordo.com.au 
>> <mailto:sa...@bordo.com.au>,10.0.0.54,): unknown user  - 
>> trying the next userdb
>> 
> 
> That is not an error. As the "Info: " prefix indicates, this is an innocuous 
> informational message.
> 
> Regards,
> 
> Stephan.
> 
Thanks Stephan.

I know it’s not a major thing, it’s just that it is every third line of my log. 

Ie I get the auth-worker unknown user, then successful imap-login, then the 
imap logout.

So I’m just trying to figure out what is causing the auth-worker lookup failure 
so I can fix it. I did not have this in my old mail server, just the new one 
I’ve setup.

Is it a database issue or a Dovecot config issue? If so where?

Thanks,

James.




"unknown user - trying the next userdb" Info in log

2019-01-28 Thread James Brown via dovecot
I’ve set up a new mail server. Auth-worker is giving me this error all the time:

auth-worker(11160): Info: passwd(sa...@bordo.com.au 
<mailto:sa...@bordo.com.au>,10.0.0.54,): unknown user  - 
trying the next userdb

Next line is from imap-login with successful login.

Using MySQL as user database.

Auth-sql.comf.ext is:

passdb {
  driver = sql

  # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
  args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
}

# "prefetch" user database means that the passdb already provided the
# needed information and there's no need to do a separate userdb lookup.
# 
userdb {
  driver = prefetch
}

userdb {
  driver = sql
  args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
}

# If you don't have any user-specific settings, you can avoid the user_query
# by using userdb static instead of userdb sql, for example:
# 
#userdb {
  #driver = static
  #args = uid=vmail gid=vmail home=/var/vmail/%u
#}

Any ideas why it is gives me this error before logging in? Where should I look?

Thanks,

James.



Re: Sieve "OOO" configuration

2019-01-05 Thread James Cassell
Look at the currentdate test https://tools.ietf.org/html/rfc5260#section-5


V/r,
James Cassell


On Sat, Jan 5, 2019, at 11:26 AM, Jerry wrote:
> I am able to get sieve issuing an "out of office"message correctly.
> However, I want to configure it to send an "OOO" message only during
> certain dates, say on weekends, or only between certain hours. I can do
> that manually; however, was wondering if there is any automatic method
> available that could handle this chore.
> 
> Thanks!
> 
> -- 
> Jerry


Re: Compiling Dovecot on Solaris 11 fails

2019-01-04 Thread James

On 03/01/2019 20:42, Sami Ketola wrote:


does work just fine on my solaris 11 x86 box.

...

my test system is solaris 11.3 and I have gcc 4.5.2.


Old versions of gcc do not have -mfunction-return=thunk

7 and 8 have it.





Re: Compiling Dovecot on Solaris 11 fails

2019-01-03 Thread James

On 03/01/2019 16:45, Andrew Watkins wrote:

Tried to build dovecot-2.3.4 on Solaris 11 x86 and it fails at configure
part. I just went checked and it last version it works on is v2.3.2.1


...


checking Linux compatible mremap()... no
checking whether shared mmaps get updated by write()s... no
checking whether fd passing works... no
configure: error: fd passing is required for Dovecot to work



In think it's thunk (but I forget exactly what aspect).

Try:
./configure --disable-hardening ...

Or build with cc.

(Sorry, it's too late for me to do another test today.)


Re: Several problems on Solaris10

2019-01-01 Thread James

On 31/12/2018 17:28, Pierluigi Frullani wrote:


> My version is 2.2.13 ( it was the last one, at the time of the first
> server setup ).

2.2.13 is from around May 2014.  It worked but I can't see why you
wouldn't switch to the latest 2.3.4. (You might be seeing what I can't
and your question hasn't explained.)


That's the date for installation. I was using the courier-imap and
switched to dovecot.
Not changed since then.


So use the latest version.



> I have seen that ( it seems ) the new solaris don't honour the
> LD_LIBRARY_PATH.

I'm sure it does but you shouldn't need it anyway.


Believe me, it doesn't :(


Test:

$ cat > sub.c << EOF
void sub() {}
EOF
$ cat > main.c << EOF
void sub();
int main() { sub(); }
EOF
$ cc -G -o libsub.so sub.c
-- link with no run path in the executable, fails with no other help
$ cc main.c -L. -lsub
$ ./a.out
ld.so.1: a.out: fatal: libsub.so: open failed: No such file or directory
zsh: killed ./a.out
-- succeeds with LD_LIBRARY_PATH set
$ LD_LIBRARY_PATH=. ./a.out
-- link with a run path, succeeds on its own
$ cc main.c -L. -R. -lsub
$ ./a.out
$ dump -Lv a.out | grep RUNPATH
[5] RUNPATH .

-- to see the logic when finding a library use "ldd -s ..."
$ LD_LIBRARY_PATH=. ldd -s ./a.out

   find object=libsub.so; required by a.out
search path=.  (LD_LIBRARY_PATH)
trying path=./libsub.so
libsub.so =>  ./libsub.so

-- clean up:
$ rm sub.c main.c libsub.so a.out



The problem ( as usual ) arise with the openssl libs that solaris ships,


Build your own openssl.



1. Do not put your files in /use/local.  You will clash with someone
else thinking it is the place to put personal stuff.  man filesystem:
"/opt  Root of a subtree for add-on application packages."


 Being that I'm the only one administering this machine I'm sure it
would not  ;)


You will appreciate the separation of system and add-ons.  Solaris 10 
sparse zones can make /usr read only.  It really is the "right" way.




Would you ( or could ) show me where to replace -R and -L on Makefile(s)
to fix the runpath trouble ?

I've tried the following:
CPPFLAGS="-I/usr/local/clucene/include -I/usr/local/libtextcat/include
-I/usr/local/openssl-1.0.1e/include" LDFLAGS="-L/usr/local/clucene/lib
-L/usr/local/libtextcat/lib
-L/usr/local/openssl-1.0.1e/lib -R/usr/local/openssl-1.0.1e/lib "
./configure --prefix=/usr/local/dovecot --with-ssl=yes --with-stemmer
--with-lucene --with-zlib
and it seems to work:

root@puma dump -Lv ./src/imap-login/.libs/imap-login | grep RUN
[14]RUNPATH
 
/usr/local/dovecot/lib/dovecot:/usr/local/lib:/usr/local/openssl-1.0.1e/lib:/usr/ccs/lib:/lib:/usr/lib:/usr/sfw/lib


It would be easier if you put openssl and your other software under a 
common prefix, $PREFIX/lib, (not a subdirectory).  The SONAME should 
sort out versions.


If you are doing a simple build you will get away with the standard 
handling of -L and -R.


$ PREFIX=/opt/PREFIX
$ export LDFLAGS="-R/opt/FIRST/lib:${PREFIX}/lib"
$ ./configure --prefix=${PREFIX} ...

Gives RPATH: /opt/PREFIX/lib/dovecot:/opt/FIRST/lib:/opt/PREFIX/lib
which I think will work for you.



Problems with libtool are when we require:
+ /opt/PREFIX/lib/$ISALIST at the front before /opt/PREFIX/lib/
+ a non standard lib is needed before the normal one instead of libtool 
using /opt/PREFIX/lib as the first


In general it is easier to let libtool think it is helping but ignore 
it.  Create a wrapper script around cc to rewrite the args back to what 
they were before configure+libtool rearranged, in this case drop the -L 
and -R given and substitute the values in $LDFLAGS.





More reading:
https://blogs.oracle.com/solaris/ldlibrarypath-just-say-no-v2
https://docs.oracle.com/cd/E86824_01/html/E54763/ld.so.1-1.html
https://docs.oracle.com/cd/E19683-01/816-1386/chapter3-13312/
Avoid crle, you don't need it and it affects the whole system.  Hint, 
you might need this to unlock your machine:

# LD_LIBRARY_PATH=/usr/lib rm /var/ld/ld.config










Re: Several problems on Solaris10

2018-12-30 Thread James

On 29/12/2018 13:49, Pierluigi Frullani wrote:


 I've just upgraded my old Solaris 10 update 8 to Solaris 10 update 11
with the latest patches, but after the reboot with the new update I'm
having  a lot of problems with dovecot.

My version is 2.2.13 ( it was the last one, at the time of the first
server setup ).


2.2.13 is from around May 2014.  It worked but I can't see why you 
wouldn't switch to the latest 2.3.4. (You might be seeing what I can't 
and your question hasn't explained.)



I have seen that ( it seems ) the new solaris don't honour the
LD_LIBRARY_PATH.


I'm sure it does but you shouldn't need it anyway.



The first error was a relocation error:
relocation error: file
/usr/local/dovecot/lib/dovecot/libdovecot-login.so.0: symbol
EVP_PKEY_get1_EC_KEY: referenced symbol not found
Just to bypass this phase I have linked the openssl-1.0.1e in
/usr/local/lib ( libssl and libcrypto ) but then I got a new relocation
error:

imap: Error: dlopen(/usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so)
failed: ld.so.1: imap: fatal: relocation error: file
/usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so: symbol
http_url_parse: referenced symbol not found

Did someone has some idea on how to bypass those troubles ?


Did you compile this yourself or are you using someone else's package? 
Solaris has no files in /usr/local, you must have added those.



1. Do not put your files in /use/local.  You will clash with someone 
else thinking it is the place to put personal stuff.  man filesystem: 
"/opt  Root of a subtree for add-on application packages."


2. Do not use LD_LIBRARY_PATH in the run time environment.  Instead use 
the runpath in binaries as set during linking.


3. On configure set the local paths
--prefix=/opt/XXX \
--sysconfdir=/etc/opt/XXX \
--localstatedir=/var/opt/XXX \
--mandir=/opt/XXX/share/man \
--docdir=/opt/XXX/share/doc/dovecot \


4. When linking use -L to point to the libraries.  These need not be in 
the installation location and during build won't be (because you haven't 
installed yet because you are building new libraries).


5. When linking use -R to point to the installed location of the libraries.


It should work if the paths are set correctly in the binaries.  A 
generic package can use $ORIGIN.
It's possible libtool is doing its usual trick of making a simple task 
difficult - I take measures to undo its wrong doing and set -L and -R 
between libtool and ld (cc -G).
I suggest building the new dovecot and getting it right but if you want 
to diagnose your current installation here are some command hints you 
might find useful.


-- find dovecot:
# find /opt -type f -name dovecot
/opt/XXX/sbin/dovecot

-- is there a package?
# pkginfo | grep dovecot
# pkginfo | nawk '/dovecot/{print $2}' | while read package
do
   pkginfo -l ${package}
done

-- what is its run path?
# dump -Lv /opt/XXX/sbin/dovecot | grep RUNPATH

-- can the runtime linker find the libraries?
# ldd -r /opt/XXX/sbin/dovecot














Re: Lda fatal: setgid (102 from userdb lookup) failed

2018-12-04 Thread James Brown via dovecot
> On 4 Dec 2018, at 5:48 pm, Aki Tuomi  wrote:
> 
> On 4.12.2018 3.27, James Brown via dovecot wrote:
>> Sorry if this is a nubbie question, but I’m getting:
>> 
>> lda(jlbr...@bordo.com.au)<>: Fatal: setgid(102 
>> from userdb lookup) failed with euid=501(jlbrown), gid=20(staff), 
>> egid=20(staff): Operation not permitted (This binary should probably be 
>> called with process group set to 102 instead of 20(staff))
>> 
>> How can I fix this?
>> 
>> (macOS Mojave, Dovecot 2.3.4)
>> 
>> Thanks,
>> 
>> James.
> 
> You need to tell your MTA to execute dovecot-lda as the target user as
> you are using per-user UIDs.
> 
> Aki

Thanks Aki.

It works now. Not sure what it was.

In Postfix’s master.cf changed the user in the dovecot line.

Also changed dovecot/auth-userdb user and group, and permissions.

Anyway, all works now. 

Thanks again,

James.


Lda fatal: setgid (102 from userdb lookup) failed

2018-12-03 Thread James Brown via dovecot
Sorry if this is a nubbie question, but I’m getting:

lda(jlbr...@bordo.com.au)<>: Fatal: setgid(102 from 
userdb lookup) failed with euid=501(jlbrown), gid=20(staff), egid=20(staff): 
Operation not permitted (This binary should probably be called with process 
group set to 102 instead of 20(staff))

How can I fix this?

(macOS Mojave, Dovecot 2.3.4)

Thanks,

James.


"Group doesn't exist: dovecot"

2018-11-19 Thread James Brown
No matter what I do, I always get this error when trying to start Dovecot:

$ sudo /usr/local/opt/dovecot/sbin/dovecot
Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 
1000), because of default_client_limit
Fatal: service(stats) Group doesn't exist: dovecot (See service stats { 
unix_listener /usr/local/var/run/dovecot/stats-writer { group } } setting)

The user and group it should be using are _dovecot.

Config:

$ sudo /usr/local/opt/dovecot/sbin/dovecot -n
# 2.3.2.1 (0719df592): /usr/local/etc/dovecot/dovecot.conf
# OS: Darwin 18.2.0 x86_64  apfs
# Hostname: mailbordocomau-dr.local
doveconf: Error: t_readlink(/usr/local/var/run/dovecot/dovecot.conf) failed: 
readlink() failed: No such file or directory
default_internal_user = _dovenull
default_login_user = _dovecot
first_valid_gid = 0
first_valid_uid = 102
last_valid_gid = 500
last_valid_uid = 105
mail_gid = 102
mail_location = maildir:/usr/local/virtual/%d/%u
mail_uid = 102
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix = 
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
protocols = imap pop3
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = _postfix
mode = 0660
user = _postfix
  }
  unix_listener auth-userdb {
mode = 0660
user = _vmail
  }
}
ssl_cert = 

Re: macOS Mojave: setgroups(501) failed: Too many extra groups

2018-11-18 Thread James Brown
> On 11 Oct 2018, at 7:35 pm, Heiko W. Rupp  wrote:
> 
> On Thu, Oct 11, 2018 at 10:55:39AM +0300, Aki Tuomi wrote:
> 
>> Maybe. Have to see when we can implement it though. It could probably
>> leverage the min/max_gid setting.
> 
> Actually that was a great hint.
> 
> Setting 
>   last_valid_gid = 100 
> in the config and restarting helped. 
> Having a filter-list instead of fixed upper/lower bounds would be more
> flexible. I guess though that in reality most systems/setups have ranges
> for different kinds of groups, so that the first/last_valid_gid could
> be applied.
> 
> Thanks
>   Heiko

Hi Heiko,

I’m also trying to set up Dovecot on a Mojave system.

Would you mind leting me know what your settings are?

Ie output of:

dovecot -n

Many thanks,

James.


Re: New install - getting error: "Failed to initialize SSL server context: Couldn't parse DH parameters"

2018-11-12 Thread James Brown
Ah… that’s better!

No error now.

Thanks Aki.

> On 13 Nov 2018, at 4:06 pm, Tuomi, Aki  wrote:
> 
> Actually you need to use ssl_dh=<
> /usr/local/etc/dovecot/dh.pem
> 
> Note the <
> 
> Aki
> 
>  Original message 
> From: "Michael A. Peters" 
> Date: 13/11/2018 05:44 (GMT+02:00)
> To: dovecot@dovecot.org
> Subject: Re: New install - getting error: "Failed to initialize SSL server 
> context: Couldn't parse DH parameters"
> 
> try
> 
> openssl dhparam -out /usr/local/etc/dovecot/dh.pem 2048
> 
> On 11/12/2018 07:28 PM, James Brown wrote:
> > I’m setting up Dovecot using Homebrew on a new server and am getting 
> > this when I try to login via IMAP:
> > 
> > Nov 13 14:13:35 auth: Debug: auth client connected (pid=30719)
> > Nov 13 14:13:35 imap-login: Info: Aborted login (no auth attempts in 0 
> > secs): user=<>, rip=::1, lip=::1, secured, 
> > session=
> > Nov 13 14:18:33 auth: Debug: Loading modules from directory: 
> > /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth
> > Nov 13 14:18:33 auth: Debug: Module loaded: 
> > /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth/lib20_auth_var_expand_crypt.so
> > Nov 13 14:18:33 auth: Debug: Read auth token secret from 
> > /usr/local/var/run/dovecot/auth-token-secret.dat
> > Nov 13 14:18:33 auth: Debug: auth client connected (pid=30848)
> > Nov 13 14:18:33 imap-login: Error: Failed to initialize SSL server 
> > context: Couldn't parse DH parameters: error:0906D06C:PEM 
> > routines:PEM_read_bio:no start line: Expecting: DH PARAMETERS: user=<>, 
> > rip=::1, lip=::1, secured, session=
> > Nov 13 14:18:33 imap-login: Info: Disconnected: TLS initialization 
> > failed. (no auth attempts in 0 secs): user=<>, rip=::1, lip=::1, 
> > secured, session=
> > 
> > I’ve used:
> > 
> > Openssl gendh 2048
> > 
> > And put the output:
> > 
> > -BEGIN DH PARAMETERS-
> > MIIBCAKCAQEA0IF7kQX32IJFm/5HEVwYf7Be4G9iY86MvLiFLL3wHGqcPT3EMsIv
> > YSe5XOT0Q7DGXPOZ+DLlJq8KDHxWKNI6j/0ZaRBrF38CWj8Jqxa8pqo9FVSWj45b
> > JwSLqBSoBIEFWibqSE6L8wlV8xjMsB34xLHduJDNbaBzsooN749CopTkmkuGeXKH
> > waOEbDzlOq+qHEa4bjx2/e/TnPj0kCrMnfeU4QILo1rJwuN4nY6k7fGwgEDVa2hE
> > oOrVfJxyuuuiblahblahblahhhXCGsxhlDQO
> > QmzOhHqPovzbBByO9iR5fu3xbNm9YRxPowIBAg==
> > -END DH PARAMETERS——
> > 
> > Into a file dh.pem and then added
> > 
> > ssl_dh=/usr/local/etc/dovecot/dh.pem
> > 
> > To my dovecot.conf file.
> > 
> > Reloaded Dovecot but still get the same error.
> > 
> > Any suggestions?
> > 
> > macOS 10.13.6, Dovecot 2.3.2.1
> > 
> > Any suggestions?
> > 
> > Thanks,
> > 
> > James.
> 




Re: New install - getting error: "Failed to initialize SSL server context: Couldn't parse DH parameters"

2018-11-12 Thread James Brown
> On 11/12/2018 07:28 PM, James Brown wrote:
>> I’m setting up Dovecot using Homebrew on a new server and am getting this 
>> when I try to login via IMAP:
>> Nov 13 14:13:35 auth: Debug: auth client connected (pid=30719)
>> Nov 13 14:13:35 imap-login: Info: Aborted login (no auth attempts in 0 
>> secs): user=<>, rip=::1, lip=::1, secured, 
>> session=
>> Nov 13 14:18:33 auth: Debug: Loading modules from directory: 
>> /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth
>> Nov 13 14:18:33 auth: Debug: Module loaded: 
>> /usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth/lib20_auth_var_expand_crypt.so
>> Nov 13 14:18:33 auth: Debug: Read auth token secret from 
>> /usr/local/var/run/dovecot/auth-token-secret.dat
>> Nov 13 14:18:33 auth: Debug: auth client connected (pid=30848)
>> Nov 13 14:18:33 imap-login: Error: Failed to initialize SSL server context: 
>> Couldn't parse DH parameters: error:0906D06C:PEM routines:PEM_read_bio:no 
>> start line: Expecting: DH PARAMETERS: user=<>, rip=::1, lip=::1, secured, 
>> session=
>> Nov 13 14:18:33 imap-login: Info: Disconnected: TLS initialization failed. 
>> (no auth attempts in 0 secs): user=<>, rip=::1, lip=::1, secured, 
>> session=
>> I’ve used:
>> Openssl gendh 2048
>> And put the output:
>> -BEGIN DH PARAMETERS-
>> MIIBCAKCAQEA0IF7kQX32IJFm/5HEVwYf7Be4G9iY86MvLiFLL3wHGqcPT3EMsIv
>> YSe5XOT0Q7DGXPOZ+DLlJq8KDHxWKNI6j/0ZaRBrF38CWj8Jqxa8pqo9FVSWj45b
>> JwSLqBSoBIEFWibqSE6L8wlV8xjMsB34xLHduJDNbaBzsooN749CopTkmkuGeXKH
>> waOEbDzlOq+qHEa4bjx2/e/TnPj0kCrMnfeU4QILo1rJwuN4nY6k7fGwgEDVa2hE
>> oOrVfJxyuuuiblahblahblahhhXCGsxhlDQO
>> QmzOhHqPovzbBByO9iR5fu3xbNm9YRxPowIBAg==
>> -END DH PARAMETERS——
>> Into a file dh.pem and then added
>> ssl_dh=/usr/local/etc/dovecot/dh.pem
>> To my dovecot.conf file.
>> Reloaded Dovecot but still get the same error.
>> Any suggestions?
>> macOS 10.13.6, Dovecot 2.3.2.1
>> Any suggestions?
>> Thanks,
>> James.
> 
> 
> On 13 Nov 2018, at 2:43 pm, Michael A. Peters  wrote:
> 
> try
> 
> openssl dhparam -out /usr/local/etc/dovecot/dh.pem 2048

Thanks Michael. Gave that a go, and it successfully created the file, I 
reloaded Dovecot, but still get the same error.

James.


New install - getting error: "Failed to initialize SSL server context: Couldn't parse DH parameters"

2018-11-12 Thread James Brown
I’m setting up Dovecot using Homebrew on a new server and am getting this when 
I try to login via IMAP:

Nov 13 14:13:35 auth: Debug: auth client connected (pid=30719)
Nov 13 14:13:35 imap-login: Info: Aborted login (no auth attempts in 0 secs): 
user=<>, rip=::1, lip=::1, secured, session=
Nov 13 14:18:33 auth: Debug: Loading modules from directory: 
/usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth
Nov 13 14:18:33 auth: Debug: Module loaded: 
/usr/local/Cellar/dovecot/2.3.2.1/lib/dovecot/auth/lib20_auth_var_expand_crypt.so
Nov 13 14:18:33 auth: Debug: Read auth token secret from 
/usr/local/var/run/dovecot/auth-token-secret.dat
Nov 13 14:18:33 auth: Debug: auth client connected (pid=30848)
Nov 13 14:18:33 imap-login: Error: Failed to initialize SSL server context: 
Couldn't parse DH parameters: error:0906D06C:PEM routines:PEM_read_bio:no start 
line: Expecting: DH PARAMETERS: user=<>, rip=::1, lip=::1, secured, 
session=
Nov 13 14:18:33 imap-login: Info: Disconnected: TLS initialization failed. (no 
auth attempts in 0 secs): user=<>, rip=::1, lip=::1, secured, 
session=

I’ve used:

Openssl gendh 2048

And put the output:

-BEGIN DH PARAMETERS-
MIIBCAKCAQEA0IF7kQX32IJFm/5HEVwYf7Be4G9iY86MvLiFLL3wHGqcPT3EMsIv
YSe5XOT0Q7DGXPOZ+DLlJq8KDHxWKNI6j/0ZaRBrF38CWj8Jqxa8pqo9FVSWj45b
JwSLqBSoBIEFWibqSE6L8wlV8xjMsB34xLHduJDNbaBzsooN749CopTkmkuGeXKH
waOEbDzlOq+qHEa4bjx2/e/TnPj0kCrMnfeU4QILo1rJwuN4nY6k7fGwgEDVa2hE
oOrVfJxyuuuiblahblahblahhhXCGsxhlDQO
QmzOhHqPovzbBByO9iR5fu3xbNm9YRxPowIBAg==
-END DH PARAMETERS——

Into a file dh.pem and then added

ssl_dh=/usr/local/etc/dovecot/dh.pem

To my dovecot.conf file.

Reloaded Dovecot but still get the same error.

Any suggestions?

macOS 10.13.6, Dovecot 2.3.2.1

Any suggestions?

Thanks,

James.

Re: Disaster Recovery Help

2018-11-12 Thread James Brown
> On 13 Nov 2018, at 9:48 am, Giovanni Bisanti  <mailto:g...@ttyllc.net>> wrote:
> 
> the Account on the iPhone is IMAP, I'm not sure I understand the process you 
> are describing
> 

iMazing might be able to help you get the mail off the iPhone:

https://imazing.com <https://imazing.com/>

Then transfer to you Dovecot Maildir. (Have never done this - it’s just a 
suggestion).

James.

Re: macOS Mojave: setgroups(501) failed: Too many extra groups

2018-10-24 Thread James Brown
On 11 Oct 2018, at 5:28 pm, Heiko W. Rupp mailto:h...@pilhuhn.de>> wrote:
> 
> Hello,
> 
> I have recently upgraded to macOS 10.14 (Mojave) and am running into an issue 
> where one use can no longer log into dovecot via imap. Log shows
> 
> Oct 11 08:10:27 imap(hwr)<12659>: Fatal: 
> setgroups(501) failed: Too many extra groups
> 
> ...
> Any help appreciated
>   Heiko

This came up on the list a few months ago. See thread with subject "dying on 
osx” on 11 August and 4 September 2018.

The only way I managed to get it to work was by changing default_login_user and 
default_internal_user to my username.

As you say, wasn’t an issue with earlier versions of macOS X.

Any solution would be most appreciated.

James.

Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-10-01 Thread James

On 01/10/2018 13:37, Stephan Bosch wrote:


I'd still like to know whether doing this helps somehow:


Does it help when you change the "> 0" at the following code position
to "!= 0" ?

https://github.com/dovecot/pigeonhole/blob/master/src/lib-sieve/sieve-binary-code.c#L300


--- ../original/src/lib-sieve/sieve-binary-code.c	2018-02-05 
19:45:53.0 +

+++ src/lib-sieve/sieve-binary-code.c   2018-10-01 14:35:18.553169973 +
@@ -297,7 +297,7 @@
return FALSE;

/* Read first integer bytes [1xxx] */
-   while ( (ADDR_DATA_AT(address) & 0x80) > 0 ) {
+   while ( (ADDR_DATA_AT(address) & 0x80) != 0 ) {
if ( ADDR_BYTES_LEFT(address) > 0 && bits > 0) {
integer |= ADDR_DATA_AT(address) & 0x7F;
ADDR_JUMP(address, 1);


Quick reply: It does. I can't see why.



James.











Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-10-01 Thread James

On 25/09/2018 22:39, Stephan Bosch wrote:

Hello,


Something mightily weird is going on at your end. It doesn't fail here


This is correct.  I believe there to be a compiler problem.  Removing 
optimisation from sieve-binary-code.c gives success.


Thank you to Stephan Bosch for testing this for me and pointing me in 
the right direction.


My debugging was confused because dovecot flags are passed to 
pigeonhole, eg $DOVECOT_CFLAGS.  This is why the build of dovecot 
affects pigeonhole.  Once I removed these I could control the pigeonhole 
build and I was able to isolate the problem.




James.




Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-10-01 Thread James

On 28/09/2018 12:38, Aki Tuomi wrote:


It was studio cc.  gcc doesn't make it through configure and I
didn't ask why.

Can you share a little bit more info on how did the compile (or
configure even) fail with gcc on Solaris 11?


$ ./configure $ARGS
...
checking Linux compatible mremap()... no
checking whether shared mmaps get updated by write()s... no
checking whether fd passing works... no
configure: error: fd passing is required for Dovecot to work

Which in the log corresponds to:

configure:22685: ./conftest
./configure[2026]: eval: line 1: 22335: Memory fault(coredump)


Appears to the option "-mfunction-return=thunk" that cause the
problem, remove and no core dump.


If you thunk that was all think again.  My gcc builds fail to link 
ending with lots of undefined symbols.  It is probably libtool 
obstructing portability but I have an easy solution of using another 
compiler and ignoring libtool.





You can use --disable-harderning to disable these things.


--disable-hardening




Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-28 Thread James

On 27/09/2018 16:14, Sami Ketola wrote:


It was studio cc.  gcc doesn't make it through configure and I didn't ask why.


Can you share a little bit more info on how did the compile (or configure even) 
fail with gcc on Solaris 11?



$ ./configure $ARGS
...
checking Linux compatible mremap()... no
checking whether shared mmaps get updated by write()s... no
checking whether fd passing works... no
configure: error: fd passing is required for Dovecot to work

Which in the log corresponds to:

configure:22685: ./conftest
./configure[2026]: eval: line 1: 22335: Memory fault(coredump)


Appears to the option "-mfunction-return=thunk" that cause the problem, 
remove and no core dump.  Older gccs do not have -mfunction-return.







as I have no problems in compiling dovecot and pigeonhole on my Solaris 11.3 
system with gcc. The version that ships with my Solaris is 4.5.2.


Strictly speaking Solaris 11 does not ship with gcc, one can install it 
[from the OS vendor] with pkg and there is a choice of versions.


# pkg list -a | grep gcc-c


I have gcc versions installed: 4.9.5, 5.5.0, 6.4.0, 7.3.0 and 8.2.0.




I also have Sun Studio 12.5 installed but I have not even tried to compile 
dovecot wit that yet.


Current Release - Oracle Developer Studio 12.6.





James.


Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-28 Thread James

On 27/09/2018 14:55, Josef 'Jeff' Sipek wrote:

On Thu, Sep 27, 2018 at 10:42:16 +0100, James wrote:

$ dovecot -c dovecot.conf -n
# 2.3.3.rc1 (14e4920d8): dovecot.conf
# Pigeonhole version 0.5.2 (7704de5e)
# OS: SunOS 5.11 i86pc


Out of curiosity, is this a Solaris system or an illumos system?


Not Illumos.
Test builds on 10 and 11.3 - RC exercise, I'll add Sparc when my machine 
is powered on.

Tested run on Solaris 10 and 11.3  - RC exercise.
Reports from 11.3, results so far the same either way.
Packages built on Solaris 10 as LCD unless there is good reason to have 
a package per rev.


Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-27 Thread James

On 27/09/2018 13:40, Stephan Bosch wrote:


Address   Line  Code
:   DEBUG BLOCK: 3
0001:   EXTENSIONS [1]:
0002: vacation
0004:2: VACATION
0007:4:   seconds: NUM 5
0009: Binary is corrupt.

The line numbers differs and 86400 is read as 5.  It is like it has
forgotten the size of an integer or is confused about endianness.
There is something strange, like an #if that guesses wrong.  At least
I have somewhere to start looking.

Thank you for checking at your end, I was worried the RC had
introduced an error and your result suggests not.  RCs are for testing
and I am.


The number is stored as a chain of bytes of which the most significant
bit indicates whether the next byte still belongs to the number. If this
bit is somehow interpreted wrong, the first byte of this number would
read as 5, thereby returning '5' as the result and ignoring subsequent
bytes (causing corruption at the next item to read).

Since you're using SunOS, your compiler may be doing something funky.
Which compiler is used anyway? Perhaps different versions for the
Dovecot releases that do and don't work?


It was studio cc.  gcc doesn't make it through configure and I didn't 
ask why.  I have some other things to do but will look at this again 
later.  Thank you for the byte code explanations.  The coding at this 
point is hard to follow with the pointers-to-functions and #defines.




James.



Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-27 Thread James

On 25/09/2018 22:39, Stephan Bosch wrote:


Something mightily weird is going on at your end. It doesn't fail here
(see below). First of all, what is your configuration (output from
`dovecot -n`)?


You have dovecot.conf but...

$ dovecot -c dovecot.conf -n
# 2.3.3.rc1 (14e4920d8): dovecot.conf
# Pigeonhole version 0.5.2 (7704de5e)
# OS: SunOS 5.11 i86pc
# Hostname: mailhost
doveconf: Warning: please set ssl_dh=doveconf: Warning: You can generate it with: dd 
if=/var/opt/xxx/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl 
dhparam -inform der > /etc/opt/xxx/dovecot/dh.pem

mail_debug = yes
mail_gid = staff
mail_location = maildir:/path/to/%d/%n/Maildir
mail_uid = james
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart extracttext

postmaster_address = postmas...@domain.tld
ssl_dh = # hidden, use -P to show it




Also, can you make a hex dump of the binary (using `sieve-dump -h
.svbin`).


As said the svbin is identical to the one create by the previous version.

Comparing the dump:
Block 0 differs because it has the source file name.
Block 1 is identical
Block 2 is identical. It is this block that is declared corrupt.




Finally, can you try to explicitly delete the binary (preferably after
preserving it elsewhere) so that it is guaranteed to be created fresh?


I did each time.




## Success at my end:


Spot the difference...


Address   Line  Code
:   DEBUG BLOCK: 3
0001:   EXTENSIONS [1]:
0002: vacation
0004:3: VACATION
0007:5:   seconds: NUM 86400


Address   Line  Code
:   DEBUG BLOCK: 3
0001:   EXTENSIONS [1]:
0002: vacation
0004:2: VACATION
0007:4:   seconds: NUM 5
0009: Binary is corrupt.

The line numbers differs and 86400 is read as 5.  It is like it has 
forgotten the size of an integer or is confused about endianness.  There 
is something strange, like an #if that guesses wrong.  At least I have 
somewhere to start looking.


Thank you for checking at your end, I was worried the RC had introduced 
an error and your result suggests not.  RCs are for testing and I am.















Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-24 Thread James

On 24/09/2018 13:35, Stephan Bosch wrote:


You can enable `-d -' and `-t - -Tlevel=matching' as well.


$ sieve-test -D -d - -t - -Tlevel=matching -c dovecot.conf sieve message
sieve-test(james): Debug: Effective uid=1001, gid=10, home=/home/james
sieve-test(james): Debug: maildir++: root=/path/to//james/Maildir, 
index=, indexpvt=, control=, inbox=/path/to//james/Maildir, alt=
sieve-test(james): Debug: sieve: Pigeonhole version 0.5.2 (7704de5e) 
initializing
sieve-test(james): Debug: sieve: include: sieve_global is not set; it is 
currently not possible to include `:global' scripts.

debug: file storage: Using Sieve script path: /home/james/sieve.
debug: file script: Opened script `sieve' from `/home/james/sieve'.
debug: Script `sieve' from /home/james/sieve successfully compiled.

* Script metadata (block: 0):

class = file
class.version = 0
location = /home/james/sieve

* Required extensions (block: 1):

  0: vacation (id: 9)

* Main program (block: 2):

Address   Line  Code
:   DEBUG BLOCK: 3
0001:   EXTENSIONS [1]:
0002: vacation
0004:2: VACATION
0007:4:   seconds: NUM 5
0009: Binary is corrupt.


  ## Started executing script 'sieve'
  VACATION: #ERROR#: unknown optional operand
   2: [[EXECUTION ABORTED]]
  ## Finished executing script 'sieve'

sieve-test(james): Info: corrupt binary deleted.
sieve-test(james): Error: unlink((null)) failed: Bad address (in 
sieve-test.c:426)
sieve-test(james): Info: final result: failed; resolved with successful 
implicit keep




Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-24 Thread James

On 24/09/2018 12:54, James wrote:

Remove 2.3.3 rc1, install 2.3.2 and 0.5.2, success with sieve-test 
including ":days 1"


Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-24 Thread James

On 24/09/2018 12:34, James wrote:


$ cat sieve
require ["vacation"];
vacation
  # Reply at most once a day to a same sender
  :days 1


...removing the ":days 1" stops the corruption:

$ sieve-test -D -c dovecot.conf sieve.2 message
sieve-test(james): Debug: Effective uid=1001, gid=10, home=/home/james
sieve-test(james): Debug: maildir++: root=/path/to//james/Maildir, 
index=, indexpvt=, control=, inbox=/path/to//james/Maildir, alt=
sieve-test(james): Debug: sieve: Pigeonhole version 0.5.3.rc1 (716b1b49) 
initializing
sieve-test(james): Debug: sieve: include: sieve_global is not set; it is 
currently not possible to include `:global' scripts.

debug: file storage: Using Sieve script path: /home/james/sieve.2.
debug: file script: Opened script `sieve.2' from `/home/james/sieve.2'.
debug: Script `sieve.2' from /home/james/sieve.2 successfully compiled.

Performed actions:

 * send vacation message:
=> seconds : 604800
=> subject : Out of office reply
=> handle  : I'm out of office, please contact Joan Doe instead.
Best regards
John DoeOut of office reply

START MESSAGE
I'm out of office, please contact Joan Doe instead.
Best regards
John Doe
END MESSAGE

Implicit keep:

 * store message in folder: INBOX

sieve-test(james): Info: final result: success




$ cat sieve.2
require ["vacation"];
vacation
  # Reply at most once a day to a same sender
# :days 1
  :subject "Out of office reply"
  # List of additional recipient addresses which are included in the 
auto replying.
  # If a mail's recipient is not the envelope recipient and it's not on 
this list,

  # no vacation reply is sent for it.
  :addresses ["j@company.dom", "john@company.dom"]
"I'm out of office, please contact Joan Doe instead.
Best regards
John Doe";



Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-24 Thread James

On 24/09/2018 11:51, Stephan Bosch wrote:


Can you try with the sieve-test command-line tool, in particular with
trace debugging enabled?


Thank you for the debugging tip, is this useful?


$ sieve-test -D -c dovecot.conf sieve message
sieve-test(james): Debug: sieve: Pigeonhole version 0.5.3.rc1 (716b1b49) 
initializing
sieve-test(james): Debug: sieve: include: sieve_global is not set; it is 
currently not possible to include `:global' scripts.

debug: file storage: Using Sieve script path: /home/james/sieve.
debug: file script: Opened script `sieve' from `/home/james/sieve'.
debug: Script `sieve' from /home/james/sieve successfully compiled.
sieve-test(james): Info: corrupt binary deleted.
sieve-test(james): Error: unlink((null)) failed: Bad address (in 
sieve-test.c:426)
sieve-test(james): Info: final result: failed; resolved with successful 
implicit keep





$ cat dovecot.conf
mail_uid = james
mail_gid = staff
mail_location = maildir:/path/to/%d/%n/Maildir
postmaster_address = postmas...@domain.tld



$ cat sieve
require ["vacation"];
vacation
  # Reply at most once a day to a same sender
  :days 1
  :subject "Out of office reply"
  # List of additional recipient addresses which are included in the 
auto replying.
  # If a mail's recipient is not the envelope recipient and it's not on 
this list,

  # no vacation reply is sent for it.
  :addresses ["j@company.dom", "john@company.dom"]
"I'm out of office, please contact Joan Doe instead.
Best regards
John Doe";



$ cat message
Subject: test
From: m...@domain.tld

testing 123


v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-24 Thread James
I have a sieve problem with dovecot v2.3.3 rc1 with either of pigeonhole 
0.5.2 or 0.5.3.rc1.  The the change that triggers this is in dovecot not 
pigeonhole.  Sieve filtering has been working with v2.3.2 and many 
earlier versions.



Log report:
Sep 24 10:36:13 mailhost dovecot: [ID 702911 mail.error] 
lmtp(u...@domain.tld)<15816>: Error: sieve: 
!!BUG!!: Binary compiled from /path/to/domain/user/dovecot.sieve is 
still corrupt; bailing out and reverting to default delivery



This is a sieve script with a vacation handler.  My other sieve filters 
are working.  My actual sieve filter worked before and was written by 
Roundcube web mail.  My failing test script is cut-n-paste from the 
sieve examples https://wiki2.dovecot.org/Pigeonhole/Sieve/Examples so 
should not be in doubt.


The text script compiles to dovecot.svbin and newly compiled files are 
identical to older versions, the compilation has not changed.




James.


  1   2   3   4   >