Re: vpopmail

[Eric Broch]

On 10/4/2018 7:27 AM, Rick Romero wrote:


Quoting Eric Broch >:




On 10/4/2018 6:34 AM, Rick Romero wrote:


Quoting Aki Tuomi >:



On 03.10.2018 23:30, Eric Broch wrote:


Hello list,

I run Dovecot with the vpopmail driver and have found that it
authenticates against the clear text password in the vpopmail
database. Is there a configuration option either at compile time, link
time, or a setting in one of the configuration files that tells the
program to authenticate against the hash instead of the clear text?


Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})
Aki



Or use SQL -  then you don't have to munge any of your tools.

password_query =
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS password, 
pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid & 
8) AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))


pw_gid refers to the the binary vpopmail flags for disable POP, IMAP, 
Webmail.


Rick

When configuring vpopmail for our purposes we use (now) the 
configuration option:


  --disable-many-domains Creates a table for each virtual domain instead of 
storing all users in a single table.
 Only valid for MySQL and PostgreSQL

This disallows (I think) the use Dovecot MySQL configuration file as every user 
is stored in a domain table of the form 'mydomain_tld'.

So, we're limited to these configurations (no dovecot-mysql.conf.ext) :

passdb {
   args = cache_key=%u webmail=127.0.0.1
   driver = vpopmail
}

userdb {
   args = cache_key=%u quota_template=quota_rule=*:backend=%q
   driver = vpopmail
}

If there is a clear text password (pw_clear_passwd) present it seems that 
Dovecot will use that instead of using the hash (pw_passwd).

It seems that in the code 'passdb-vpopmail.c' (below) that if the clear 
password (pw_clear_passwd) is present Dovecot skips the hashed password 
(pw_passwd), and we want authentication against the hashed password.


 if (vpopmail_is_disabled(auth_request, vpw)) {
 auth_request_log_info(auth_request, AUTH_SUBSYS_DB,
   "%s disabled in vpopmail for this user",
   auth_request->service);
 password = NULL;
 *result_r = PASSDB_RESULT_USER_DISABLED;
 } else {
 if (vpw->pw_clear_passwd != NULL &&
 *vpw->pw_clear_passwd != '\0') {
 password = t_strdup_noconst(vpw->pw_clear_passwd);
 *cleartext = TRUE;
 } else if (!*cleartext)
 password = t_strdup_noconst(vpw->pw_passwd);
 else
 password = NULL;
 *result_r = password != NULL ? PASSDB_RESULT_OK :
 PASSDB_RESULT_SCHEME_NOT_AVAILABLE;
 }



Looking for an option to make dovecot use hashed password instead of clear text.

Hope this makes sense.

-EricB

We seem to have lost quoting..
First - Why aren't you just deleting all the clear text passwords?

Second, for many domanis, my password query for your purposes should just be:
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS password, pw_dir 
as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM %d WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid & 8) AND !(pw_gid & 2) AND 
('%r'!='' or !(pw_gid & 4))

Where %d is the domain name. Your vpopmail database should have a bunch of 
domain.com table names.
Or you can hardcode the database with   FROM vpopmail.%d
You may need to play with quotes..  FROM `vpopmail.%d`  or  FROM `%d`

Rick

I think we're getting to the crux.

The domain tables are not in 'domain.com' form but in 'domain_com' form 
(who knows why the vpopmail authors would do that)...users being in the 
table.
So, if I add a virtual domain (our vpopmail configuration 
'--disable-many-domains') to my mail server, e.g., 'mydomain.com', 
vpopmail creates a domain table 'mydomain_com', the '.' (dot) replaced 
by an '_' (underscore).
That's why the SQL query above will not work. If there were a way (and I 
don't know it) to replace the '.' with an '_' in the query we could go 
that way. Until then we have to simply use the current dovecot vpopmail 
driver

instead of the more robust sql driver.

Again, the vpopmail driver will use the clear text password (if present) 
and not bother with the hashed password.


One solution as you mentioned was to delete the clear text, but I simply 
wanted to avoid messing with the database and implement a more simple 
configuration option.


Thank you, Rick

--EricB

--
Eric Broch
White Horse Technical Consulting (WHTC)

Re: vpopmail

[Aki Tuomi]

> On 04 October 2018 at 17:42 Rick Romero  wrote:
> 
> 
>   Quoting Rick Romero :
> 
> > Quoting Eric Broch :
> >
> >> On 10/4/2018 7:27 AM, Rick Romero wrote:
> >>> Quoting Eric Broch  >>> >:
> >>>
>  On 10/4/2018 6:34 AM, Rick Romero wrote:
> >  
> >>>
> >>> Quoting Aki Tuomi  >>> >:
> >>>
>  On 03.10.2018 23:30, Eric Broch wrote:
> 
> > Hello list,
> >
> > I run Dovecot with the vpopmail driver and have found that it
> > authenticates against the clear text password in the vpopmail
> > database. Is there a configuration option either at compile time, link
> > time, or a setting in one of the configuration files that tells the
> > program to authenticate against the hash instead of the clear text?
> 
>  Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})
>  Aki
> >>>
> >>> Or use SQL -  then you don't have to munge any of your tools.
> >>>
> >>> password_query =
> >>> SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS  
> >>> password, pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
> >>> FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND  
> >>> !(pw_gid & 8) AND !(pw_gid & 2) AND ('%r'!='' or  
> >>> !(pw_gid & 4))
> >>>
> >>> pw_gid refers to the the binary vpopmail flags for disable POP,  
> >>> IMAP, Webmail.
> >>>
> >>> Rick
> >>
> >>> When configuring vpopmail for our purposes we use (now) the  
> >>> configuration option:
> >>>
> >>> --disable-many-domains     Creates a table for each virtual domain  
> >>> instead of storing all users in a single table.
> >>>                            Only valid for MySQL and PostgreSQL
> >>>
> >>> This disallows (I think) the use Dovecot MySQL configuration file  
> >>> as every user is stored in a domain table of the form  
> >>> 'mydomain_tld'.
> >>>
> >>> So, we're limited to these configurations (no dovecot-mysql.conf.ext) :
> >>>
> >>> passdb {
> >>> args = cache_key=%u webmail=127.0.0.1
> >>> driver = vpopmail
> >>> }
> >>>
> >>> userdb {
> >>> args = cache_key=%u quota_template=quota_rule=*:backend=%q
> >>> driver = vpopmail
> >>> }
> >>>
> >>> If there is a clear text password (pw_clear_passwd) present it  
> >>> seems that Dovecot will use that instead of using the hash  
> >>> (pw_passwd).
> >>>
> >>> It seems that in the code 'passdb-vpopmail.c' (below) that if the  
> >>> clear password (pw_clear_passwd) is present Dovecot skips the  
> >>> hashed password (pw_passwd), and we want authentication against  
> >>> the hashed password.
> >>>
> >>> 
> >>>        if (vpopmail_is_disabled(auth_request, vpw)) {
> >>>                auth_request_log_info(auth_request, AUTH_SUBSYS_DB,
> >>>                                      "%s disabled in vpopmail for  
> >>> this user",
> >>>                                      auth_request->service);
> >>>                password = NULL;
> >>>                *result_r = PASSDB_RESULT_USER_DISABLED;
> >>>        } else {
> >>>                if (vpw->pw_clear_passwd != NULL &&
> >>>                    *vpw->pw_clear_passwd != '\0') {
> >>>                        password = t_strdup_noconst(vpw->pw_clear_passwd);
> >>>                        *cleartext = TRUE;
> >>>                } else if (!*cleartext)
> >>>                        password = t_strdup_noconst(vpw->pw_passwd);
> >>>                else
> >>>                        password = NULL;
> >>>                *result_r = password != NULL ? PASSDB_RESULT_OK :
> >>>                        PASSDB_RESULT_SCHEME_NOT_AVAILABLE;
> >>>        }
> >>> 
> >>>
> >>> Looking for an option to make dovecot use hashed password instead  
> >>> of clear text.
> >>>
> >>> Hope this makes sense.
> >>>
> >>> -EricB
> >>>
> >>> We seem to have lost quoting..
> >>> First - Why aren't you just deleting all the clear text passwords?
> >>>
> >>> Second, for many domanis, my password query for your purposes  
> >>> should just be:
> >>> SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS  
> >>> password, pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
> >>> FROM %d WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid &  
> >>> 8) AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))
> >>>
> >>> Where %d is the domain name. Your vpopmail database should have a  
> >>> bunch of domain.com table names.
> >>> Or you can hardcode the database with   FROM vpopmail.%d
> >>> You may need to play with quotes..  FROM `vpopmail.%d`  or  FROM `%d`
> >>>
> >>> Rick
> >>
> >> Rick,
> >>
> >> I'm not sure what you're saying.
> >>
> >> Vpopmail's DB can be configured in two different ways, 1) With  
> >> domain tables and all users for that particular domain underneath  
> >> (described below), or 2) Simply, one table with all users with the  
> >> domain field 'pw_domain' (This works with dovecot-sql.conf.ext  
> >> files). The former (1), which we use does not allow the use of  
> >> dovecot-sql.conf.ext files, we're 

Re: vpopmail

[Rick Romero]

 Quoting Rick Romero :


Quoting Eric Broch :


On 10/4/2018 7:27 AM, Rick Romero wrote:
Quoting Eric Broch >:



On 10/4/2018 6:34 AM, Rick Romero wrote:

 


Quoting Aki Tuomi >:



On 03.10.2018 23:30, Eric Broch wrote:


Hello list,

I run Dovecot with the vpopmail driver and have found that it
authenticates against the clear text password in the vpopmail
database. Is there a configuration option either at compile time, link
time, or a setting in one of the configuration files that tells the
program to authenticate against the hash instead of the clear text?


Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})
Aki


Or use SQL -  then you don't have to munge any of your tools.

password_query =
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS  
password, pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND  
!(pw_gid & 8) AND !(pw_gid & 2) AND ('%r'!='' or  
!(pw_gid & 4))


pw_gid refers to the the binary vpopmail flags for disable POP,  
IMAP, Webmail.


Rick


When configuring vpopmail for our purposes we use (now) the  
configuration option:


--disable-many-domains     Creates a table for each virtual domain  
instead of storing all users in a single table.

                           Only valid for MySQL and PostgreSQL

This disallows (I think) the use Dovecot MySQL configuration file  
as every user is stored in a domain table of the form  
'mydomain_tld'.


So, we're limited to these configurations (no dovecot-mysql.conf.ext) :

passdb {
args = cache_key=%u webmail=127.0.0.1
driver = vpopmail
}

userdb {
args = cache_key=%u quota_template=quota_rule=*:backend=%q
driver = vpopmail
}

If there is a clear text password (pw_clear_passwd) present it  
seems that Dovecot will use that instead of using the hash  
(pw_passwd).


It seems that in the code 'passdb-vpopmail.c' (below) that if the  
clear password (pw_clear_passwd) is present Dovecot skips the  
hashed password (pw_passwd), and we want authentication against  
the hashed password.



       if (vpopmail_is_disabled(auth_request, vpw)) {
               auth_request_log_info(auth_request, AUTH_SUBSYS_DB,
                                     "%s disabled in vpopmail for  
this user",

                                     auth_request->service);
               password = NULL;
               *result_r = PASSDB_RESULT_USER_DISABLED;
       } else {
               if (vpw->pw_clear_passwd != NULL &&
                   *vpw->pw_clear_passwd != '\0') {
                       password = t_strdup_noconst(vpw->pw_clear_passwd);
                       *cleartext = TRUE;
               } else if (!*cleartext)
                       password = t_strdup_noconst(vpw->pw_passwd);
               else
                       password = NULL;
               *result_r = password != NULL ? PASSDB_RESULT_OK :
                       PASSDB_RESULT_SCHEME_NOT_AVAILABLE;
       }


Looking for an option to make dovecot use hashed password instead  
of clear text.


Hope this makes sense.

-EricB

We seem to have lost quoting..
First - Why aren't you just deleting all the clear text passwords?

Second, for many domanis, my password query for your purposes  
should just be:
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS  
password, pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM %d WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid &  
8) AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))


Where %d is the domain name. Your vpopmail database should have a  
bunch of domain.com table names.

Or you can hardcode the database with   FROM vpopmail.%d
You may need to play with quotes..  FROM `vpopmail.%d`  or  FROM `%d`

Rick


Rick,

I'm not sure what you're saying.

Vpopmail's DB can be configured in two different ways, 1) With  
domain tables and all users for that particular domain underneath  
(described below), or 2) Simply, one table with all users with the  
domain field 'pw_domain' (This works with dovecot-sql.conf.ext  
files). The former (1), which we use does not allow the use of  
dovecot-sql.conf.ext files, we're limited to userdb and passwd  
options previously mentioned. When using these options dovecot will  
get the clear text password if present.


The problem is that if a password is over 16 characters long the  
clear text field will only store the first 16 characters while the  
hashed field will contain the whole password.


# echo "describe domain_tld" | mysql -u root -p`cat vpoppasswd` vpopmail
yeilds
Field   Type    Null    Key     Default Extra
pw_name char(32)        NO      PRI     NULL
pw_passwd       char(40)        YES             NULL
pw_uid  int(11) YES             NULL
pw_gid  int(11) YES             NULL
pw_gecos        char(48)        YES             NULL
pw_dir  char(160)       YES             NULL
pw_shell        char(20)        YES             

Re: vpopmail

[Rick Romero]

Quoting Eric Broch :


On 10/4/2018 7:27 AM, Rick Romero wrote:


Quoting Eric Broch >:




On 10/4/2018 6:34 AM, Rick Romero wrote:


Quoting Aki Tuomi >:



On 03.10.2018 23:30, Eric Broch wrote:


Hello list,

I run Dovecot with the vpopmail driver and have found that it
authenticates against the clear text password in the vpopmail
database. Is there a configuration option either at compile time, link
time, or a setting in one of the configuration files that tells the
program to authenticate against the hash instead of the clear text?


Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})
Aki



Or use SQL -  then you don't have to munge any of your tools.

password_query =
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS  
password, pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND  
!(pw_gid & 8) AND !(pw_gid & 2) AND ('%r'!='' or  
!(pw_gid & 4))


pw_gid refers to the the binary vpopmail flags for disable POP,  
IMAP, Webmail.


Rick



When configuring vpopmail for our purposes we use (now) the  
configuration option:


 --disable-many-domains Creates a table for each virtual domain  
instead of storing all users in a single table.

Only valid for MySQL and PostgreSQL

This disallows (I think) the use Dovecot MySQL configuration file  
as every user is stored in a domain table of the form 'mydomain_tld'.


So, we're limited to these configurations (no dovecot-mysql.conf.ext) :

passdb {
  args = cache_key=%u webmail=127.0.0.1
  driver = vpopmail
}

userdb {
  args = cache_key=%u quota_template=quota_rule=*:backend=%q
  driver = vpopmail
}

If there is a clear text password (pw_clear_passwd) present it  
seems that Dovecot will use that instead of using the hash  
(pw_passwd).


It seems that in the code 'passdb-vpopmail.c' (below) that if the  
clear password (pw_clear_passwd) is present Dovecot skips the  
hashed password (pw_passwd), and we want authentication against the  
hashed password.



if (vpopmail_is_disabled(auth_request, vpw)) {
auth_request_log_info(auth_request, AUTH_SUBSYS_DB,
  "%s disabled in vpopmail for  
this user",

  auth_request->service);
password = NULL;
*result_r = PASSDB_RESULT_USER_DISABLED;
} else {
if (vpw->pw_clear_passwd != NULL &&
*vpw->pw_clear_passwd != '\0') {
password = t_strdup_noconst(vpw->pw_clear_passwd);
*cleartext = TRUE;
} else if (!*cleartext)
password = t_strdup_noconst(vpw->pw_passwd);
else
password = NULL;
*result_r = password != NULL ? PASSDB_RESULT_OK :
PASSDB_RESULT_SCHEME_NOT_AVAILABLE;
}



Looking for an option to make dovecot use hashed password instead  
of clear text.


Hope this makes sense.

-EricB

We seem to have lost quoting..
First - Why aren't you just deleting all the clear text passwords?

Second, for many domanis, my password query for your purposes  
should just be:
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS  
password, pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM %d WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid & 8)  
AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))


Where %d is the domain name. Your vpopmail database should have a  
bunch of domain.com table names.

Or you can hardcode the database with   FROM vpopmail.%d
You may need to play with quotes..  FROM `vpopmail.%d`  or  FROM `%d`

Rick


Rick,

I'm not sure what you're saying.

Vpopmail's DB can be configured in two different ways, 1) With  
domain tables and all users for that particular domain underneath  
(described below), or 2) Simply, one table with all users with the  
domain field 'pw_domain' (This works with dovecot-sql.conf.ext  
files). The former (1), which we use does not allow the use of  
dovecot-sql.conf.ext files, we're limited to userdb and passwd  
options previously mentioned. When using these options dovecot will  
get the clear text password if present.


The problem is that if a password is over 16 characters long the  
clear text field will only store the first 16 characters while the  
hashed field will contain the whole password.


# echo "describe domain_tld" | mysql -u root -p`cat vpoppasswd` vpopmail
yeilds
Field   TypeNullKey Default Extra
pw_name char(32)NO  PRI NULL
pw_passwd   char(40)YES NULL
pw_uid  int(11) YES NULL
pw_gid  int(11) YES NULL
pw_gecoschar(48)YES NULL
pw_dir  char(160)   YES NULL
pw_shellchar(20)YES

Re: VOLATILEDIR not really used?

[Tom Sommer]

On 2018-10-04 15:55, Timo Sirainen wrote:

On 4 Oct 2018, at 14.39, Tom Sommer  wrote:


Is this correct, and if so are there any plans to move dotlocks etc. 
to this directory?



What dotlocks? I guess mbox and Maildir have some locks that could be
moved there, but a better performance optimization for those
installations would be to switch to sdbox/mdbox. Other than that, I
don't remember there being anything important that could be moved
there.


Maildir locks yes, switching format is not a procedure I feel 
comfortable with :)


Would it be possible to move the maildir/mbox locks to VOLATILEDIR?

--
Tom Sommer

Re: vpopmail

[Eric Broch]

On 10/4/2018 7:27 AM, Rick Romero wrote:


Quoting Eric Broch >:




On 10/4/2018 6:34 AM, Rick Romero wrote:


Quoting Aki Tuomi >:



On 03.10.2018 23:30, Eric Broch wrote:


Hello list,

I run Dovecot with the vpopmail driver and have found that it
authenticates against the clear text password in the vpopmail
database. Is there a configuration option either at compile time, link
time, or a setting in one of the configuration files that tells the
program to authenticate against the hash instead of the clear text?


Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})
Aki



Or use SQL -  then you don't have to munge any of your tools.

password_query =
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS password, 
pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid & 
8) AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))


pw_gid refers to the the binary vpopmail flags for disable POP, IMAP, 
Webmail.


Rick


Rick,

I'm not sure what you're saying.

Vpopmail's DB can be configured in two different ways, 1) With domain 
tables and all users for that particular domain underneath (described 
below), or 2) Simply, one table with all users with the domain field 
'pw_domain' (This works with dovecot-sql.conf.ext files). The former 
(1), which we use does not allow the use of dovecot-sql.conf.ext files, 
we're limited to userdb and passwd options previously mentioned. When 
using these options dovecot will get the clear text password if present.


The problem is that if a password is over 16 characters long the clear 
text field will only store the first 16 characters while the hashed 
field will contain the whole password.


# echo "describe domain_tld" | mysql -u root -p`cat vpoppasswd` vpopmail
yeilds
Field   Type    Null    Key Default Extra
pw_name char(32)    NO  PRI NULL
pw_passwd   char(40)    YES NULL
pw_uid  int(11) YES NULL
pw_gid  int(11) YES NULL
pw_gecos    char(48)    YES NULL
pw_dir  char(160)   YES NULL
pw_shell    char(20)    YES NULL
pw_clear_passwd char(16)    YES NULL

As you can see there is no 'pw_domain' field from which to draw.

Again we are limited to the passdb, and userdb options already described.


When configuring vpopmail for our purposes we use (now) the 
configuration option:


  --disable-many-domains Creates a table for each virtual domain instead of 
storing all users in a single table.
 Only valid for MySQL and PostgreSQL

This disallows (I think) the use Dovecot MySQL configuration file as every user 
is stored in a domain table of the form 'mydomain_tld'.

So, we're limited to these configurations (no dovecot-mysql.conf.ext) :

passdb {
   args = cache_key=%u webmail=127.0.0.1
   driver = vpopmail
}

userdb {
   args = cache_key=%u quota_template=quota_rule=*:backend=%q
   driver = vpopmail
}

If there is a clear text password (pw_clear_passwd) present it seems that 
Dovecot will use that instead of using the hash (pw_passwd).

It seems that in the code 'passdb-vpopmail.c' (below) that if the clear 
password (pw_clear_passwd) is present Dovecot skips the hashed password 
(pw_passwd), and we want authentication against the hashed password.


 if (vpopmail_is_disabled(auth_request, vpw)) {
 auth_request_log_info(auth_request, AUTH_SUBSYS_DB,
   "%s disabled in vpopmail for this user",
   auth_request->service);
 password = NULL;
 *result_r = PASSDB_RESULT_USER_DISABLED;
 } else {
 if (vpw->pw_clear_passwd != NULL &&
 *vpw->pw_clear_passwd != '\0') {
 password = t_strdup_noconst(vpw->pw_clear_passwd);
 *cleartext = TRUE;
 } else if (!*cleartext)
 password = t_strdup_noconst(vpw->pw_passwd);
 else
 password = NULL;
 *result_r = password != NULL ? PASSDB_RESULT_OK :
 PASSDB_RESULT_SCHEME_NOT_AVAILABLE;
 }



Looking for an option to make dovecot use hashed password instead of clear text.

Hope this makes sense.

-EricB

We seem to have lost quoting..
First - Why aren't you just deleting all the clear text passwords?

Second, for many domanis, my password query for your purposes should just be:
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS password, pw_dir 
as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM %d WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid & 8) AND !(pw_gid & 2) AND 
('%r'!='' or !(pw_gid & 4))

Where %d is the domain name. Your vpopmail database should have a bunch 

Re: VOLATILEDIR not really used?

[Timo Sirainen]

On 4 Oct 2018, at 14.39, Tom Sommer  wrote:
> 
> Hi
> 
> According to the docs, setting VOLATILEDIR will improve I/O performance when 
> using NFS - but as far as I can see, only vsize lock-files are put here, and 
> little else?

Right. I think also autoexpunge locking.

> Is this correct, and if so are there any plans to move dotlocks etc. to this 
> directory?


What dotlocks? I guess mbox and Maildir have some locks that could be moved 
there, but a better performance optimization for those installations would be 
to switch to sdbox/mdbox. Other than that, I don't remember there being 
anything important that could be moved there.

Re: vpopmail

[Rick Romero]

 Quoting Eric Broch :


On 10/4/2018 6:34 AM, Rick Romero wrote:


 


 Quoting Aki Tuomi :


On 03.10.2018 23:30, Eric Broch wrote:


Hello list,

I run Dovecot with the vpopmail driver and have found that it
authenticates against the clear text password in the vpopmail
database. Is there a configuration option either at compile time, link
time, or a setting in one of the configuration files that tells the
program to authenticate against the hash instead of the clear text?


Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})
Aki



Or use SQL -  then you don't have to munge any of your tools.

password_query =
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS password,  
pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid &  
8) AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))


pw_gid refers to the the binary vpopmail flags for disable POP, IMAP, Webmail.

Rick
When configuring vpopmail for our purposes we use (now) the  
configuration option:


 --disable-many-domains Creates a table for each virtual domain  
instead of storing all users in a single table. 
 Only valid for MySQL and PostgreSQL  This disallows (I think)  
the use Dovecot MySQL configuration file as every user is stored in a  
domain table of the form 'mydomain_tld'.  So, we're limited to these  
configurations (no dovecot-mysql.conf.ext) :  passdb {   args =  
cache_key=%u webmail=127.0.0.1   driver = vpopmail }  userdb {   args  
= cache_key=%u quota_template=quota_rule=*:backend=%q   driver =  
vpopmail }  If there is a clear text password (pw_clear_passwd)  
present it seems that Dovecot will use that instead of using the hash  
(pw_passwd).  It seems that in the code 'passdb-vpopmail.c' (below)  
that if the clear password (pw_clear_passwd) is present Dovecot skips  
the hashed password (pw_passwd), and we want authentication against  
the hashed password.  if  
(vpopmail_is_disabled(auth_request, vpw)) {  
auth_request_log_info(auth_request, AUTH_SUBSYS_DB, 
   "%s disabled in vpopmail for this user",
auth_request->service);  
password = NULL; *result_r =  
PASSDB_RESULT_USER_DISABLED; } else { if  
(vpw->pw_clear_passwd != NULL &&  
*vpw->pw_clear_passwd != '\0') { password =  
t_strdup_noconst(vpw->pw_clear_passwd);  
*cleartext = TRUE; } else if (!*cleartext)  
password = t_strdup_noconst(vpw->pw_passwd);   
   else password = NULL;  
*result_r = password != NULL ? PASSDB_RESULT_OK :   
   PASSDB_RESULT_SCHEME_NOT_AVAILABLE; }Looking  
for an option to make dovecot use hashed password instead of clear  
text.   Hope this makes sense.  -EricB  We seem to have lost quoting..  
First - Why aren't you just deleting all the clear text passwords?   
Second, for many domanis, my password query for your purposes should  
just be: SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS  
password, pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid  
FROM %d WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid & 8)  
AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))  Where  
%d is the domain name. Your vpopmail database should have a bunch of  
domain.com table names. Or you can hardcode the database with   FROM  
vpopmail.%d  You may need to play with quotes..  FROM `vpopmail.%d`   
or  FROM `%d`   Rick

Re: vpopmail

[Eric Broch]

On 10/4/2018 6:34 AM, Rick Romero wrote:


Quoting Aki Tuomi >:



On 03.10.2018 23:30, Eric Broch wrote:


Hello list,

I run Dovecot with the vpopmail driver and have found that it
authenticates against the clear text password in the vpopmail
database. Is there a configuration option either at compile time, link
time, or a setting in one of the configuration files that tells the
program to authenticate against the hash instead of the clear text?


Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})
Aki



Or use SQL -  then you don't have to munge any of your tools.

password_query =
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS password, 
pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid & 
8) AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))


pw_gid refers to the the binary vpopmail flags for disable POP, IMAP, 
Webmail.


Rick

When configuring vpopmail for our purposes we use (now) the 
configuration option:


 --disable-many-domains Creates a table for each virtual domain instead of 
storing all users in a single table.
Only valid for MySQL and PostgreSQL

This disallows (I think) the use Dovecot MySQL configuration file as every user 
is stored in a domain table of the form 'mydomain_tld'.

So, we're limited to these configurations (no dovecot-mysql.conf.ext) :

passdb {
  args = cache_key=%u webmail=127.0.0.1
  driver = vpopmail
}

userdb {
  args = cache_key=%u quota_template=quota_rule=*:backend=%q
  driver = vpopmail
}

If there is a clear text password (pw_clear_passwd) present it seems that 
Dovecot will use that instead of using the hash (pw_passwd).

It seems that in the code 'passdb-vpopmail.c' (below) that if the clear 
password (pw_clear_passwd) is present Dovecot skips the hashed password 
(pw_passwd), and we want authentication against the hashed password.


if (vpopmail_is_disabled(auth_request, vpw)) {
auth_request_log_info(auth_request, AUTH_SUBSYS_DB,
  "%s disabled in vpopmail for this user",
  auth_request->service);
password = NULL;
*result_r = PASSDB_RESULT_USER_DISABLED;
} else {
if (vpw->pw_clear_passwd != NULL &&
*vpw->pw_clear_passwd != '\0') {
password = t_strdup_noconst(vpw->pw_clear_passwd);
*cleartext = TRUE;
} else if (!*cleartext)
password = t_strdup_noconst(vpw->pw_passwd);
else
password = NULL;
*result_r = password != NULL ? PASSDB_RESULT_OK :
PASSDB_RESULT_SCHEME_NOT_AVAILABLE;
}



Looking for an option to make dovecot use hashed password instead of clear text.

Hope this makes sense.

-EricB






--
Eric Broch
White Horse Technical Consulting (WHTC)

Re: [ceph-users] dovecot + cephfs - sdbox vs mdbox

[Webert de Souza Lima]

Hi, bring this up again to ask one more question:

what would be the best recommended locking strategy for dovecot against
cephfs?
this is a balanced setup using independent director instances but all
dovecot instances on each node share the same storage system (cephfs).

Regards,

Webert Lima
DevOps Engineer at MAV Tecnologia
*Belo Horizonte - Brasil*
*IRC NICK - WebertRLZ*


On Wed, May 16, 2018 at 5:15 PM Webert de Souza Lima 
wrote:

> Thanks Jack.
>
> That's good to know. It is definitely something to consider.
> In a distributed storage scenario we might build a dedicated pool for that
> and tune the pool as more capacity or performance is needed.
>
> Regards,
>
> Webert Lima
> DevOps Engineer at MAV Tecnologia
> *Belo Horizonte - Brasil*
> *IRC NICK - WebertRLZ*
>
>
> On Wed, May 16, 2018 at 4:45 PM Jack  wrote:
>
>> On 05/16/2018 09:35 PM, Webert de Souza Lima wrote:
>> > We'll soon do benchmarks of sdbox vs mdbox over cephfs with bluestore
>> > backend.
>> > We'll have to do some some work on how to simulate user traffic, for
>> writes
>> > and readings. That seems troublesome.
>> I would appreciate seeing these results !
>>
>> > Thanks for the plugins recommendations. I'll take the change and ask you
>> > how is the SIS status? We have used it in the past and we've had some
>> > problems with it.
>>
>> I am using it since Dec 2016 with mdbox, with no issue at all (I am
>> currently using Dovecot 2.2.27-3 from Debian Stretch)
>> The only config I use is mail_attachment_dir, the rest lies as default
>> (mail_attachment_min_size = 128k, mail_attachment_fs = sis posix,
>> ail_attachment_hash = %{sha1})
>> The backend storage is a local filesystem, and there is only one Dovecot
>> instance
>>
>> >
>> > Regards,
>> >
>> > Webert Lima
>> > DevOps Engineer at MAV Tecnologia
>> > *Belo Horizonte - Brasil*
>> > *IRC NICK - WebertRLZ*
>> >
>> >
>> > On Wed, May 16, 2018 at 4:19 PM Jack  wrote:
>> >
>> >> Hi,
>> >>
>> >> Many (most ?) filesystems does not store multiple files on the same
>> block
>> >>
>> >> Thus, with sdbox, every single mail (you know, that kind of mail with
>> 10
>> >> lines in it) will eat an inode, and a block (4k here)
>> >> mdbox is more compact on this way
>> >>
>> >> Another difference: sdbox removes the message, mdbox does not : a
>> single
>> >> metadata update is performed, which may be packed with others if many
>> >> files are deleted at once
>> >>
>> >> That said, I do not have experience with dovecot + cephfs, nor have
>> made
>> >> tests for sdbox vs mdbox
>> >>
>> >> However, and this is a bit out of topic, I recommend you look at the
>> >> following dovecot's features (if not already done), as they are awesome
>> >> and will help you a lot:
>> >> - Compression (classic, https://wiki.dovecot.org/Plugins/Zlib)
>> >> - Single-Instance-Storage (aka sis, aka "attachment deduplication" :
>> >> https://www.dovecot.org/list/dovecot/2013-December/094276.html)
>> >>
>> >> Regards,
>> >> On 05/16/2018 08:37 PM, Webert de Souza Lima wrote:
>> >>> I'm sending this message to both dovecot and ceph-users ML so please
>> >> don't
>> >>> mind if something seems too obvious for you.
>> >>>
>> >>> Hi,
>> >>>
>> >>> I have a question for both dovecot and ceph lists and below I'll
>> explain
>> >>> what's going on.
>> >>>
>> >>> Regarding dbox format (https://wiki2.dovecot.org/MailboxFormat/dbox),
>> >> when
>> >>> using sdbox, a new file is stored for each email message.
>> >>> When using mdbox, multiple messages are appended to a single file
>> until
>> >> it
>> >>> reaches/passes the rotate limit.
>> >>>
>> >>> I would like to understand better how the mdbox format impacts on IO
>> >>> performance.
>> >>> I think it's generally expected that fewer larger file translate to
>> less
>> >> IO
>> >>> and more troughput when compared to more small files, but how does
>> >> dovecot
>> >>> handle that with mdbox?
>> >>> If dovecot does flush data to storage upon each and every new email is
>> >>> arrived and appended to the corresponding file, would that mean that
>> it
>> >>> generate the same ammount of IO as it would do with one file per
>> message?
>> >>> Also, if using mdbox many messages will be appended to a said file
>> >> before a
>> >>> new file is created. That should mean that a file descriptor is kept
>> open
>> >>> for sometime by dovecot process.
>> >>> Using cephfs as backend, how would this impact cluster performance
>> >>> regarding MDS caps and inodes cached when files from thousands of
>> users
>> >> are
>> >>> opened and appended all over?
>> >>>
>> >>> I would like to understand this better.
>> >>>
>> >>> Why?
>> >>> We are a small Business Email Hosting provider with bare metal, self
>> >> hosted
>> >>> systems, using dovecot for servicing mailboxes and cephfs for email
>> >> storage.
>> >>>
>> >>> We are currently working on dovecot and storage redesign to be in
>> >>> production ASAP. The main objective is to serve more users with better
>> >>> performance, high 

Re: vpopmail

[Rick Romero]

 Quoting Aki Tuomi :


On 03.10.2018 23:30, Eric Broch wrote:

Hello list,

I run Dovecot with the vpopmail driver and have found that it
authenticates against the clear text password in the vpopmail
database. Is there a configuration option either at compile time, link
time, or a setting in one of the configuration files that tells the
program to authenticate against the hash instead of the clear text?


Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})
Aki


Or use SQL -  then you don't have to munge any of your tools.

password_query =
SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS password,  
pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid
FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid &  
8) AND !(pw_gid & 2) AND ('%r'!='' or !(pw_gid & 4))


pw_gid refers to the the binary vpopmail flags for disable POP, IMAP, Webmail.

Rick

VOLATILEDIR not really used?

[Tom Sommer]

Hi

According to the docs, setting VOLATILEDIR will improve I/O performance 
when using NFS - but as far as I can see, only vsize lock-files are put 
here, and little else?


Is this correct, and if so are there any plans to move dotlocks etc. to 
this directory?


Thanks.
--
Tom

Re: Username aliases

[Eirik Rye]

On 26/09/2018 08:34, Aki Tuomi wrote:
> # before current passbd
> passdb {
>driver = passwd-file
>args = username_format=%Lu /etc/dovecot/aliases
> }
>
> # into /etc/dovecot/aliases
> alias@user:::user=real_username noauthenticate
>
> This hopefully works.
>
> Aki

This does appear to work, thanks!

As an optimization of the alias file, we tried setting "override_fields 
= noauthenticate" in the passdb instead of placing it as an extra field 
for every entry, however this does not work.


Setting "default_fields = noauthenticate" does work, however.

Why is this? According to the documentation, default_fields and 
override_fields are the same, except the latter one will override fields 
returned by the passdb? I would expect either configuration to set the 
"noauthenticate" flag.


Does override_fields only override fields that have already been set?

Finally, the dovecot debug log logs the statement "Allowing any 
password" before logging "Not performing authentication (noauthenticate 
set)" which sounds somewhat ominous, but this may just be a product of 
the way "noauthenticate" is implemented?


--
Eirik

Re: Bug reports - auth is broken in Dovecot 2.3.3

[Florian Pritz]

On Thu, Oct 04, 2018 at 11:40:05AM +0300, Aki Tuomi 
 wrote:
> The patch is already in master, but 2.3.3 is not released as 'HEAD'
> from master. The reason it's not in 2.3.3, is that we do not see it as
> super-urgent patch, since it affects glibc-2.2.28 mostly. 

Thanks!

Florian


signature.asc
Description: PGP signature

Re: Bug reports - auth is broken in Dovecot 2.3.3

[Aki Tuomi]

> On 04 October 2018 at 11:05 Florian Pritz  wrote:
> 
> 
> On Wed, Oct 03, 2018 at 03:47:11PM +0300, Berindeie Avram-Teodor 
>  wrote:
> > On Wed, Oct 3, 2018 at 2:44 PM Timo Sirainen  wrote:
> > > On 3 Oct 2018, at 14.09, Berindeie Avram-Teodor 
> > > wrote:
> > >> On Wed, Oct 3, 2018 at 1:53 PM Timo Sirainen  wrote:
> > >>> On 3 Oct 2018, at 13.22, Berindeie Avram-Teodor 
> > >>> 
> > >>> wrote:
> 
> This seems to be a lot of discussion about a rather simple patch. What
> was the reason for not including it in the 2.3.3 release, given that the
> patch date says it is from August? To me it seems like a really simple
> patch that shouldn't cause any problems so I would have expected it to
> be included into master right away, but maybe I'm wrong there and there
> is some hidden issue. If that's the case, please tell me since I'd like
> to understand the reasoning a bit better.
> 
> Florian

The patch is already in master, but 2.3.3 is not released as 'HEAD' from 
master. The reason it's not in 2.3.3, is that we do not see it as super-urgent 
patch, since it affects glibc-2.2.28 mostly. 

Aki

Re: Bug reports - auth is broken in Dovecot 2.3.3

[Florian Pritz]

On Wed, Oct 03, 2018 at 03:47:11PM +0300, Berindeie Avram-Teodor 
 wrote:
> On Wed, Oct 3, 2018 at 2:44 PM Timo Sirainen  wrote:
> > On 3 Oct 2018, at 14.09, Berindeie Avram-Teodor 
> > wrote:
> >> On Wed, Oct 3, 2018 at 1:53 PM Timo Sirainen  wrote:
> >>> On 3 Oct 2018, at 13.22, Berindeie Avram-Teodor 
> >>> wrote:

This seems to be a lot of discussion about a rather simple patch. What
was the reason for not including it in the 2.3.3 release, given that the
patch date says it is from August? To me it seems like a really simple
patch that shouldn't cause any problems so I would have expected it to
be included into master right away, but maybe I'm wrong there and there
is some hidden issue. If that's the case, please tell me since I'd like
to understand the reasoning a bit better.

Florian


signature.asc
Description: PGP signature

Re: Dovecot User Listing Error - getpwent() failed: Invalid Argument

[Reuben Farrelly]

As a followup to this thread, it is looking increasingly like this is a 
regression in glibc-2.28.  One of the Gentoo devs has reproduced the 
problem, created a test case and bisected it down to a specific commit 
that went into glibc in July.


For more information see:

https://bugs.gentoo.org/667118

Note: it's not a problem with glibc-2.27, but new to 2.28.

Reuben


On 19/09/2018 6:53 pm, Aki Tuomi wrote:

I have no idea why it's returning invalid argument, maybe you need to
raise this as issue with glibc?

Aki


On 19.09.2018 10:43, Reuben Farrelly wrote:

Sure:

thunderstorm ~ # cat /etc/nsswitch.conf
# /etc/nsswitch.conf:
# $Header:
/var/cvsroot/gentoo/src/patchsets/glibc/extra/etc/nsswitch.conf,v 1.2
2017/08/12 16:21:44 slyfox Exp $

passwd:  compat files
shadow:  compat files
group:   compat files

hosts:   files dns
networks:    files dns

services:    db files
protocols:   db files
rpc: db files
ethers:  db files
netmasks:    files
netgroup:    files
bootparams:  files

automount:   files
aliases: files
thunderstorm ~ #

Reuben

On 19/09/2018 5:41 pm, Aki Tuomi wrote:

Can you provide your /etc/nssswitch.conf?

Aki


On 19.09.2018 10:40, Reuben Farrelly wrote:

I've been running with this almost identical patch already for some
time:

diff -up dovecot-2.3.0.1/src/auth/mycrypt.c.libxcrypt
dovecot-2.3.0.1/src/auth/mycrypt.c
--- dovecot-2.3.0.1/src/auth/mycrypt.c.libxcrypt   2018-02-28
15:28:58.0 +0100
+++ dovecot-2.3.0.1/src/auth/mycrypt.c 2018-03-27 10:57:38.447769201
+0200
@@ -14,6 +14,7 @@
   #  define _XPG6 /* Some Solaris versions require this, some break
with this */
   #endif
   #include 
+#include 

   #include "mycrypt.h"

That fixed the build problem I had, but the warning below remains.

(Tracking master-2.3 git)

Reuben


On 19/09/2018 5:34 pm, Aki Tuomi wrote:

This is bit of a longshot, but can you try applying
https://github.com/dovecot/core/commit/63a74b9.patch and see if this
helps?

Aki


On 19.09.2018 10:33, Reuben Farrelly wrote:

glibc-2.28.

There was a posting by Michael Marley on 31/08 saying he saw
something
similar to this problem after upgrading to 2.28 as well with 'doveadm
purge'.

Reuben

On 19/09/2018 4:50 pm, Aki Tuomi wrote:

Which glibc version are you using?

Aki

On 19.09.2018 09:39, Reuben Farrelly wrote:

Hi Aki,

How did you go with this in the end?  Do you need anything more
from
me or were you able to reproduce this?

Thanks,
Reuben

On 1/09/2018 12:41 am, Reuben Farrelly wrote:

Sure:

https://www.reub.net/files/dovecot/lightning-dovecot.conf
https://www.reub.net/files/dovecot/thunderstorm-dovecot.conf

Updated nightly.

Reuben


On 1/09/2018 12:26 am, Aki Tuomi wrote:

Can you provide doveconf -n?



---
Aki Tuomi
Dovecot oy

 Original message 
From: Reuben Farrelly 
Date: 31/08/2018 17:12 (GMT+02:00)
To: Aki Tuomi , Dovecot Mailing List

Subject: Re: Dovecot User Listing Error - getpwent() failed:
Invalid
Argument

No.  Neither of those are installed on either system.

Reuben

On 1/09/2018 12:09 am, Aki Tuomi wrote:
 > Could apparmor or selinux be causing this!
 >
 >
 >
 > ---
 > Aki Tuomi
 > Dovecot oy
 >
 >  Original message 
 > From: Reuben Farrelly 
 > Date: 31/08/2018 16:50 (GMT+02:00)
 > To: Dovecot Mailing List 
 > Subject: Dovecot User Listing Error - getpwent() failed:
Invalid
Argument
 >
 > Hi,
 >
 > I'm running dovecot-2.3 git and seeing on 2 of my Dovecot
installations,
 > the following message logged quite frequently:
 >
 > Aug 31 16:55:53 lightning.reub.net dovecot[7698]:
auth-worker(7707):
 > Error: getpwent() failed: Invalid argument
 > Aug 31 16:55:53 lightning.reub.net dovecot[7698]:
replicator:
Error:
 > User listing returned failure
 > Aug 31 16:55:53 lightning.reub.net dovecot[7698]:
replicator:
Error:
 > listing users failed, can't replicate existing data
 >
 > It appears that this could be having an effect on
replication,
but it's
 > unclear because the secondary/replica doesn't see much
traffic.
 >
 > Authentication is via PAM and the system is Gentoo Linux
x86_64.
passb
 > driver = pam, userdb driver = static .
 >
 > doveadm user -u '*' shows the system users listed twice
like this
(is
 > this a problem?) and then prints:
 >
 > 
 > reuben
 > liam
 > reuben
 > liam
 > 
 > Error: User listing returned failure
 > Fatal: user listing failed
 > 
 >
 > To me that doesn't look right.
 >
 > Can anyone suggest what could be causing this?  I don't
recall
seeing
 > this message some time ago so I suspect it's a recent
change in
either
 > dovecot, or pam or glibc etc..
 >
 > Thanks,
 > Reuben

Re: vpopmail

[Aki Tuomi]

On 03.10.2018 23:30, Eric Broch wrote:
> Hello list,
>
> I run Dovecot with the vpopmail driver and have found that it
> authenticates against the clear text password in the vpopmail
> database. Is there a configuration option either at compile time, link
> time, or a setting in one of the configuration files that tells the
> program to authenticate against the hash instead of the clear text?
>

Prefix your passwords in vpopmail with {SCHEME} (like,  {CRYPT})

Aki