Re: doveadm user '*' not working, virtual users only with sqlite
> "John" == John Stoffel writes:
Do I think I'm on the right track here, since I removed the following
from /etc/dovecot/conf.d/auth-sql.conf.ext
#userdb {
# driver = static
# args = uid=mail gid=mail home=/var/mail/%d/%n
#}
So now my error is as follows:
# doveadm user -u '*'
Error: auth-master: userdb list: User listing returned failure
Fatal: user listing failed
Because now when I restart dovecot, I see the following in the log:
Dec 01 16:55:14 master: Info: Dovecot v2.3.21 (47349e2482) starting up
for imap, lmtp, sieve (core dumps disabled)
Dec 01 16:55:14 auth: Warning: sql: Ignoring changed iterate_query in
/etc/dovecot/dovecot-sql.conf.ext, because userdb sql not used. (If
this is intentional, set userdb_warning_disable=yes)
Dec 01 16:55:14 auth: Error: auth-master client: Trying to iterate
users, but userdbs don't support it (created 0 msecs ago, handshake 0
msecs ago)
So I commented out my 'iterate_query = ...' (see below) from
/etc/postfix/dovecot-sql.conf.ext and now I get the error on startup
which says:
Dec 01 16:57:42 master: Info: Dovecot v2.3.21 (47349e2482) starting up
for imap, lmtp, sieve (core dumps disabled)
Dec 01 16:57:42 auth: Error: auth-master client: Trying to iterate
users, but userdbs don't support it (created 0 msecs ago, handshake 0
msecs ago)
Dec 01 16:57:42 replicator: Error: auth-master: userdb list: User
listing returned failure
Dec 01 16:57:42 replicator: Error: listing users failed, can't
replicate existing data
Which tells me I need the iteracte_users setting, but I've got a bogus
query in there. So I think I should be using something like this:
iterate_query = SELECT email AS user from virtual_users;
where 'virtual_users' is the one and only table in my sqlite db file.
And I'm just returning the 'email' column as 'user', since that's what
it seems to expect.
Hmmm...
> I've been pounding my head against the sand for a while here trying to
> figure out why I can't get:
>doveadm user '*'
> working properly. I've got a Debian 11 VPS runnig dovecot version
> 2.3.21-1+debian10 and it works great. But not I'm trying to add in
> simple replication to a home dovecot instance over a wireguard tunnel
> so I can do backups and have a little better resiliency. Maybe.
> In any case, my sqlite schema looks like this:
sqlite> .schema virtual_users
> CREATE TABLE `virtual_users` (
> `id` integer NOT NULL PRIMARY KEY AUTOINCREMENT
> , `domain_id` integer NOT NULL
> , `password` varchar(106) NOT NULL
> , `email` varchar(100) NOT NULL
> , UNIQUE (`email`)
> , CONSTRAINT `virtual_users_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES
> `virtual_domains` (`id`) E
> );
> CREATE INDEX "idx_virtual_users_domain_id" ON "virtual_users"
> (`domain_id`);
> and I don't have any other tables. The 'domain_id' was/is a leftover
> from my thinking I needed it for extra testing of other domains and
> such.
> I can do 'doveadm user j...@stoffel.org' and it works just fine. When
> I do "doveadm user '*'" it fails and I get:
> doveadm user '*'
> Error: auth-master: userdb list: User listing returned failure
> Fatal: user listing failed
> So my config looks like this:
>root@mail:/etc/dovecot/conf.d# cat auth-sql.conf.ext
># Authentication for SQL users. Included from 10-auth.conf.
>#
>#
>passdb {
> driver = sql
> # Path for SQL configuration file, see
>example-config/dovecot-sql.conf.ext
> args = /etc/dovecot/dovecot-sql.conf.ext
>}
>userdb {
> driver = static
> args = uid=mail gid=mail home=/var/mail/%d/%n
>}
> My /etc/dovecot/dovecot-sql.conf.ext has the following:
>driver = sqlite
>connect = /etc/dovecot/private/virtual_users.sqlite3
>default_pass_scheme = SHA512-CRYPT
>password_query = SELECT '/var/mail/%d/%u' AS userdb_home, 'mail' AS
> userdb_uid, 'mail' AS userdb_gid, email as user, password FROM virtual_users
> WHERE email='%u';
>iterate_query = SELECT email AS user from virtual_users;
> And my general doveadm config output is this, slightly edited down to
> remove stuff I don't think I need to show is at the end. Any hints on
> what I've done wrong here? Do I need a more complete sqlite3 schema?
> I wish I could get more debugging info on what query it's trying to
> run and the error(s) it's getting.
> Thanks,
> John
> # 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.21 (f6cd4b8e)
> # OS: Linux 5.10.0-26-amd64 x86_64 Debian 11.8 ext4
> # Hostname: localhost
> # NOTE: Send doveconf -n output instead when asking for help.
> auth_anonymous_username = anonymous
> auth_cache_negative_ttl = 1 hours
> auth_cache_size = 0
> auth_cache_ttl = 1 hours
> auth_cache_verify_password_with_worker = no
> auth_debug = no
> auth_debug_passwords = no
> auth_failure_delay = 2
doveadm user '*' not working, virtual users only with sqlite
Hi all,
I've been pounding my head against the sand for a while here trying to
figure out why I can't get:
doveadm user '*'
working properly. I've got a Debian 11 VPS runnig dovecot version
2.3.21-1+debian10 and it works great. But not I'm trying to add in
simple replication to a home dovecot instance over a wireguard tunnel
so I can do backups and have a little better resiliency. Maybe.
In any case, my sqlite schema looks like this:
sqlite> .schema virtual_users
CREATE TABLE `virtual_users` (
`id` integer NOT NULL PRIMARY KEY AUTOINCREMENT
, `domain_id` integer NOT NULL
, `password` varchar(106) NOT NULL
, `email` varchar(100) NOT NULL
, UNIQUE (`email`)
, CONSTRAINT `virtual_users_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES
`virtual_domains` (`id`) E
);
CREATE INDEX "idx_virtual_users_domain_id" ON "virtual_users" (`domain_id`);
and I don't have any other tables. The 'domain_id' was/is a leftover
from my thinking I needed it for extra testing of other domains and
such.
I can do 'doveadm user j...@stoffel.org' and it works just fine. When
I do "doveadm user '*'" it fails and I get:
doveadm user '*'
Error: auth-master: userdb list: User listing returned failure
Fatal: user listing failed
So my config looks like this:
root@mail:/etc/dovecot/conf.d# cat auth-sql.conf.ext
# Authentication for SQL users. Included from 10-auth.conf.
#
#
passdb {
driver = sql
# Path for SQL configuration file, see
example-config/dovecot-sql.conf.ext
args = /etc/dovecot/dovecot-sql.conf.ext
}
userdb {
driver = static
args = uid=mail gid=mail home=/var/mail/%d/%n
}
My /etc/dovecot/dovecot-sql.conf.ext has the following:
driver = sqlite
connect = /etc/dovecot/private/virtual_users.sqlite3
default_pass_scheme = SHA512-CRYPT
password_query = SELECT '/var/mail/%d/%u' AS userdb_home, 'mail' AS
userdb_uid, 'mail' AS userdb_gid, email as user, password FROM virtual_users
WHERE email='%u';
iterate_query = SELECT email AS user from virtual_users;
And my general doveadm config output is this, slightly edited down to
remove stuff I don't think I need to show is at the end. Any hints on
what I've done wrong here? Do I need a more complete sqlite3 schema?
I wish I could get more debugging info on what query it's trying to
run and the error(s) it's getting.
Thanks,
John
# 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.21 (f6cd4b8e)
# OS: Linux 5.10.0-26-amd64 x86_64 Debian 11.8 ext4
# Hostname: localhost
# NOTE: Send doveconf -n output instead when asking for help.
auth_anonymous_username = anonymous
auth_cache_negative_ttl = 1 hours
auth_cache_size = 0
auth_cache_ttl = 1 hours
auth_cache_verify_password_with_worker = no
auth_debug = no
auth_debug_passwords = no
auth_failure_delay = 2 secs
auth_gssapi_hostname =
auth_krb5_keytab =
auth_master_user_separator =
auth_mechanisms = plain login
auth_policy_check_after_auth = yes
auth_policy_check_before_auth = yes
auth_policy_hash_mech = sha256
auth_policy_hash_nonce =
auth_policy_hash_truncate = 12
auth_policy_log_only = no
auth_policy_reject_on_fail = no
auth_policy_report_after_auth = yes
auth_policy_request_attributes = login=%{requested_username}
pwhash=%{hashed_password} remote=%{rip} device_id=%{client_id} protocol=%s
session_id=%{session}
auth_policy_server_api_header =
auth_policy_server_timeout_msecs = 2000
auth_policy_server_url =
auth_proxy_self =
auth_realms =
auth_socket_path = auth-userdb
auth_ssl_require_client_cert = no
auth_ssl_username_from_cert = no
auth_stats = no
auth_use_winbind = no
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_username_format = %Lu
auth_username_translation =
auth_verbose = no
auth_verbose_passwords = no
auth_winbind_helper_path = /usr/bin/ntlm_auth
auth_worker_max_count = 30
base_dir = /run/dovecot
config_cache_size = 1 M
debug_log_path =
default_client_limit = 1000
default_idle_kill = 1 mins
default_internal_group = dovecot
default_internal_user = dovecot
default_login_user = dovenull
default_process_limit = 100
default_vsz_limit = 256 M
deliver_log_format = msgid=%m: %$
dict_db_config =
disable_plaintext_auth = yes
dotlock_use_excl = yes
doveadm_allowed_commands =
doveadm_api_key =
doveadm_http_rawlog_dir =
doveadm_password =
doveadm_port = 0
doveadm_socket_path = doveadm-server
doveadm_ssl = no
doveadm_username = doveadm
doveadm_worker_count = 0
first_valid_gid = 1
first_valid_uid = 0
import_environment = TZ CORE_OUTOFMEM CORE_ERROR LISTEN_PID LISTEN_FDS
NOTIFY_SOCKET
info_log_path =
libexec_dir = /usr/lib/dovecot
listen = *
log_core_filter =
log_debug =
log_path = /var/log/dovecot.log
log_timestamp = "%b %d %H:%M:%S "
mail_access_groups =
mail_always_cache_fields =
mail_attachment_detection_options =
mail_attachment_dir =
mail_attachment_fs = sis posix
mail_attachment_hash =
auth_mechanisms per listener or local_name?
Hi all, I would like to use kerberos authentication on the local network but not for connections from internet, which are forwarded by haproxy. So both types of login can be distinguished by a different inet_listener and also a different by local_name. Is there a way to set auth_mechanisms such the it will impossible to use kerberos via haproxy connections or by filtering on local_name? - Kees. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: doveadm backup has problems with some accounts
> On 01/12/2023 11:34 EET roger.meier--- via dovecot > wrote: > > > I try to setup for our mailserver (in a dc) a job backup to a secondary > server (in our office) with "doveadm backup" via the -R option. (rsync , > rsnapshot and so on tooks too long at the moment) > > For the most user accounts all work as expected, but for three users it > always fails > > First run: > /usr/bin/doveadm -Dv backup -Ru user1 tcp:mua01.domain.intra:12345 > > > Dec 01 08:54:21 doveadm(user1)<70126>: Debug: > > Mailbox INBOX: Mailbox opened because: copy caching decisions > > Dec 01 08:54:22 doveadm(user1)<70126>: Warning: > > Deleting mailbox 'Archiv.Allgemein unsortiert': UID=1 > > GUID=1350559209.P2052Q54.lin-mail1.intra is missing locally > > Dec 01 08:54:22 doveadm(user1)<70126>: Debug: brain > > M: Import Archiv.Allgemein unsortiert: Import change type=save > > GUID=1350559209.P2052Q54.mail1.intra UID=1 > hdr_hash= result=Reverting > > local change by deleting mailbox - No more local mails found > > Dec 01 08:54:22 doveadm(70126): Debug: brain M: Import Archiv.Allgemein > > unsortiert: Saved UIDs: > > Dec 01 08:54:22 doveadm(70126): Debug: auth-master: conn > > unix:/run/dovecot/auth-userdb (pid=661,uid=0): Disconnected: Connection > > closed (fd=9) > > So it aborts on the lines above and a incomplete maildir is on the local > server. so the next doveadm try say's that he can't delete INBOX , etc. > > > Dec 01 09:03:33 doveadm(user1)<70169>: Error: > > Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted. > > This a very old account on our server which was migrated over different > iterations of our dovecot mailserver. The user behind the account uses > outlook over years. > Is the space here a problem? > > Another user with a similiar issue has the following error message at the end: > > > Dec 01 09:22:14 doveadm(user2)<70354>: Debug: > > Mailbox INBOX: Mailbox opened because: copy caching decisions > > Dec 01 09:22:56 doveadm(user2)<70354>: Error: > > read(mua01.domain.intra) failed: EOF (last sent=mailbox, last recv=mailbox) > > Dec 01 09:22:56 doveadm(70354): Debug: auth-master: conn > > unix:/run/dovecot/auth-userdb (pid=661,uid=0): Disconnected: Connection > > closed (fd=9) > > I see that the user has a very big inbox? when i check in the source server > in the "cur" folder of the INBOX it count's around 269372 mails. Is this > perhaps the issue for this message? > > And on the last user it looks like: > > Dec 01 09:25:19 doveadm(user3)<70376>: Warning: > Deleting mailbox 'Entwürfe': UID=1265 > GUID=1521832311.M6830P7652.mail-mx1.intra,S=1764,W=1809 is missing locally > Dec 01 09:25:19 doveadm(user3)<70376>: Debug: brain > M: Import Entwürfe: Import change type=save > GUID=1521832311.M6830P7652.mail-mx1.intra,S=1764,W=1809 UID=1265 hdr_hash= > result=Reverting local change by deleting mailbox - No more local mails found > Dec 01 09:25:19 doveadm(70376): Debug: brain M: Import Entwürfe: Saved UIDs: > Dec 01 09:25:19 doveadm(70376): Debug: auth-master: conn > unix:/run/dovecot/auth-userdb (pid=661,uid=0): Disconnected: Connection > closed (fd=9) > > The "Entwürfe" folder directly cannot be the issue? On the filesystem the > folder has the name ".Entw" and also other users has similar folders > in their mailboxes. > > The source server has dovecot version 2.2.36 (centos 7.9.2009) and the > current destination server dovecot version 2.3.16 (ubuntu 22.04 lts) Try deleting the target mailbox completely. This usually sorts these kind of things out. Also please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/ Aki ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
doveadm backup has problems with some accounts
I try to setup for our mailserver (in a dc) a job backup to a secondary server (in our office) with "doveadm backup" via the -R option. (rsync , rsnapshot and so on tooks too long at the moment) For the most user accounts all work as expected, but for three users it always fails First run: /usr/bin/doveadm -Dv backup -Ru user1 tcp:mua01.domain.intra:12345 > Dec 01 08:54:21 doveadm(user1)<70126>: Debug: Mailbox > INBOX: Mailbox opened because: copy caching decisions > Dec 01 08:54:22 doveadm(user1)<70126>: Warning: > Deleting mailbox 'Archiv.Allgemein unsortiert': UID=1 > GUID=1350559209.P2052Q54.lin-mail1.intra is missing locally > Dec 01 08:54:22 doveadm(user1)<70126>: Debug: brain > M: Import Archiv.Allgemein unsortiert: Import change type=save > GUID=1350559209.P2052Q54.mail1.intra UID=1 > hdr_hash= result=Reverting local > change by deleting mailbox - No more local mails found > Dec 01 08:54:22 doveadm(70126): Debug: brain M: Import Archiv.Allgemein > unsortiert: Saved UIDs: > Dec 01 08:54:22 doveadm(70126): Debug: auth-master: conn > unix:/run/dovecot/auth-userdb (pid=661,uid=0): Disconnected: Connection > closed (fd=9) So it aborts on the lines above and a incomplete maildir is on the local server. so the next doveadm try say's that he can't delete INBOX , etc. > Dec 01 09:03:33 doveadm(user1)<70169>: Error: Mailbox > INBOX sync: mailbox_delete failed: INBOX can't be deleted. This a very old account on our server which was migrated over different iterations of our dovecot mailserver. The user behind the account uses outlook over years. Is the space here a problem? Another user with a similiar issue has the following error message at the end: > Dec 01 09:22:14 doveadm(user2)<70354>: Debug: Mailbox > INBOX: Mailbox opened because: copy caching decisions > Dec 01 09:22:56 doveadm(user2)<70354>: Error: > read(mua01.domain.intra) failed: EOF (last sent=mailbox, last recv=mailbox) > Dec 01 09:22:56 doveadm(70354): Debug: auth-master: conn > unix:/run/dovecot/auth-userdb (pid=661,uid=0): Disconnected: Connection > closed (fd=9) I see that the user has a very big inbox? when i check in the source server in the "cur" folder of the INBOX it count's around 269372 mails. Is this perhaps the issue for this message? And on the last user it looks like: Dec 01 09:25:19 doveadm(user3)<70376>: Warning: Deleting mailbox 'Entwürfe': UID=1265 GUID=1521832311.M6830P7652.mail-mx1.intra,S=1764,W=1809 is missing locally Dec 01 09:25:19 doveadm(user3)<70376>: Debug: brain M: Import Entwürfe: Import change type=save GUID=1521832311.M6830P7652.mail-mx1.intra,S=1764,W=1809 UID=1265 hdr_hash= result=Reverting local change by deleting mailbox - No more local mails found Dec 01 09:25:19 doveadm(70376): Debug: brain M: Import Entwürfe: Saved UIDs: Dec 01 09:25:19 doveadm(70376): Debug: auth-master: conn unix:/run/dovecot/auth-userdb (pid=661,uid=0): Disconnected: Connection closed (fd=9) The "Entwürfe" folder directly cannot be the issue? On the filesystem the folder has the name ".Entw" and also other users has similar folders in their mailboxes. The source server has dovecot version 2.2.36 (centos 7.9.2009) and the current destination server dovecot version 2.3.16 (ubuntu 22.04 lts) ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
