Hi Aki, Dovecot developers
so this is a bug in dovecot 2.4 which prevents iteration over 1000s of
users with doveadm and -A flag. The workaround as identified by Philip
is to do iteration outside dovecot and then feed that into doveadm with
-u flag.
Maybe it's not such a big issue for Dovecot CE where user bases may be
low enough not to run into it. Could be of more of interest for large
users bases and Dovecot Pro. The effect is that doveadm quota recalc and
index does not terminate correctly.
It was working on dovecot 2.3 and the code hasn't really undergone that
much change.
My assumption is that in auth/auth-master-connection.c, in the
master_input_list_callback() function where there is a call
o_stream_flush(ctx->conn->output), that this call no longer appears to
reduce the o_stream_get_buffer_used_size under the MAX_OUTBUF_SIZE limit
and so get into a case with the code did not handle correctly.
This patch seems to fix it in Dovecot 2.4 by always
calling userdb_blocking_iter_next(ctx->iter) regardless of used buffer
size. I admit though that this may not be the right fix, if the
difference in the way o_stream_flush works is the root cause.
--- auth/auth-master-connection.c.orig 2026-03-08 18:06:28.025317016
+0000
+++ auth/auth-master-connection.c 2026-03-08 18:07:08.590631101 +0000
@@ -570,10 +570,9 @@
master_input_list_finish(ctx);
return;
}
- if (o_stream_get_buffer_used_size(ctx->conn->conn.output) <
MAX_OUTBUF_SIZE)
- userdb_blocking_iter_next(ctx->iter);
- else
+ if (o_stream_get_buffer_used_size(ctx->conn->conn.output) >=
MAX_OUTBUF_SIZE)
o_stream_uncork(ctx->conn->conn.output);
+ userdb_blocking_iter_next(ctx->iter);
}
static int master_input_list(struct auth_master_connection *conn,
John
On 01/03/2026 16:36, John Fawcett via dovecot wrote:
Hi
I can confirm the issue of these iteration operation not completing for
1000's of users is linked to:
MAX_OUTBUF_SIZE defined in auth/auth-master-connection.c
I made it 1024000 and managed to complete quota recalc for 10000 users.
But that's not going to be a solution, since whatever it is set to
provides a hard coded limit to how many users can be iterated for quota
get, quota recalc and index operations. Surprisingly it is not a
completely deterministic limit, since sometimes more users get processed
than other times. I was looking for a time based limit before I thought of
looking at the buffer size.
The strange thing is that the current value hasn't changed in more than 20
years if ever, so that does not explain of itself why this issue arose now
in 2.4.
#define MAX_OUTBUF_SIZE (1024*50)
The likely explanation is that exceeding the buffer size is now being
handled differently to before.
John
_______________________________________________
dovecot mailing list -- [email protected]
To unsubscribe send an email to [email protected]
