Re: Dovecot correct ownership for logs

[Michael Kjörling]

On 19 May 2024 11:11 -0400, from monn...@iro.umontreal.ca (Stefan Monnier):
>> If you have read permission on a directory but *not* execute permissions,
>> then the only thing you can do is read the contents of that directory --
>> the filenames and their inode numbers.  You cannot stat() the files,
>> so you can't see who owns them or even what kind of files they are.
>> Just their names.
> 
> Never found a situation where this as useful.

Put the user that a HTTP server runs in into each user's group, and
set g=x on each user's home directory plus ~/public_html. (Your choice
whether you want g=rx or g=x on the latter.) The web server process is
now able to access web content from within the user's home directory
without needing to run anything as root (possibly after dropping
privileges after binding to ports 80 and 443), and all this without
giving anyone else any access into other users' home directories.

Sure, something like www.example.com/~username mapping to a directory
inside username's home directory isn't that common a pattern these
days, but it used to be _very_ common. And since it's exposed more or
less directly to untrusted network inputs, it's nice to be able to
make the HTTP server drop privileges as much as possible.

This generalizes to any process having reason to descend into a
directory but not reason to enumerate the contents of the directory it
needs to descend into. Another example would be a process wanting to
manage its own /var/log/$subsystem directory itself; it doesn't need
to do anything to anything in /var/log, it only needs to be able to
descend into its own directory.

Yes, you _can_ do it in other ways. But the above is definitely _one_
way.

-- 
Michael Kjörling  https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

Re: Dovecot correct ownership for logs

[Greg Wooledge]

On Sun, May 19, 2024 at 05:15:40PM +0200, Richard wrote:
> Then where does the combination rwx come in here? With read the app knows
> the file is there, with write it writes to the file. Question is, where the
> necessity would be to know the owner of the file or even the kind. The
> logger is supposed to just append text to a file.

Stop trying to reason out WHY things are the way they are.  Just accept it.

You need execute permission on a directory in order to access anything
within that directory.  That's how it is.  That's the decision the
creators of Unix came up with.

In order for a program to write (append) to a file that already exists,
it needs:

 * execute permission on every directory leading up to that file
 * write permission on the file

If the file does NOT already exist, then the program needs to be able to
create it, and in that case, it will need:

 * execute permission on every directory leading up to where the file goes
 * write permission on the final (leaf) directory

File creation, file removal, and file renaming are directory operations,
and they need write (plus execute, of course) permission on the directory.

Modifying an existing file is a file operation, so that needs write
permission on the file (plus execute permission on the directory).

In EVERY case, you always need execute permission on all the parent
directories as well.  This is well-understood, so we usually don't bother
saying it explicitly.

Kudos to whoever you spoke with on the Dovecot list who thought to go
all the way back to first principles on this one.  We got distracted
with all of the other complex ways that this setup could have been
incorrect, and forgot to check one of the most basic ones.

Re: Dovecot correct ownership for logs

[Richard]

Then where does the combination rwx come in here? With read the app knows
the file is there, with write it writes to the file. Question is, where the
necessity would be to know the owner of the file or even the kind. The
logger is supposed to just append text to a file. If it were trying to
append text to something it doesn't have the ownership for, it should just
get an error, as that would ultimately be the case when it tries to write
to the file knowing it doesn't have ownership because the user said so. No
need for further knowledge.

Am So., 19. Mai 2024 um 17:04 Uhr schrieb Greg Wooledge :

> On Sun, May 19, 2024 at 04:55:09PM +0200, Richard wrote:
> > Dovecot expects execution permissions on the directory it writes the logs
> > to. Because "Standard POSIX permissions for a non-root process to enter a
> > directory." How on earth is that even a thing?
>
> That's how Unix permissions have always worked.  In order to access
> a file, you need +x permissions on *all* of the directories leading
> up to that file, and then appropriate permissions on the file itself.
>
> If you have read permission on a directory but *not* execute permissions,
> then the only thing you can do is read the contents of that directory --
> the filenames and their inode numbers.  You cannot stat() the files,
> so you can't see who owns them or even what kind of files they are.
> Just their names.
>
> If you have execute permission but *not* read permission on a directory,
> then you can access the files within the directory, but only if you
> already know their names.  You can't read the directory to get their
> names.
>
> Likewise, write permission on a directory allows you to rename or unlink
> files contained within that directory (because the names are not a
> property of the files -- they are part of the *directory*).  You don't
> need write permission on a file to unlink it.  Only on the directory.
>
>

Re: Dovecot correct ownership for logs

[Stefan Monnier]

> If you have read permission on a directory but *not* execute permissions,
> then the only thing you can do is read the contents of that directory --
> the filenames and their inode numbers.  You cannot stat() the files,
> so you can't see who owns them or even what kind of files they are.
> Just their names.

Never found a situation where this as useful.

> If you have execute permission but *not* read permission on a directory,
> then you can access the files within the directory, but only if you
> already know their names.  You can't read the directory to get their
> names.

This OTOH is very handy, making the filename into a kind of "passwd" to
access the file's content.


Stefan

Re: Dovecot correct ownership for logs

[Greg Wooledge]

On Sun, May 19, 2024 at 04:55:09PM +0200, Richard wrote:
> Dovecot expects execution permissions on the directory it writes the logs
> to. Because "Standard POSIX permissions for a non-root process to enter a
> directory." How on earth is that even a thing?

That's how Unix permissions have always worked.  In order to access
a file, you need +x permissions on *all* of the directories leading
up to that file, and then appropriate permissions on the file itself.

If you have read permission on a directory but *not* execute permissions,
then the only thing you can do is read the contents of that directory --
the filenames and their inode numbers.  You cannot stat() the files,
so you can't see who owns them or even what kind of files they are.
Just their names.

If you have execute permission but *not* read permission on a directory,
then you can access the files within the directory, but only if you
already know their names.  You can't read the directory to get their
names.

Likewise, write permission on a directory allows you to rename or unlink
files contained within that directory (because the names are not a
property of the files -- they are part of the *directory*).  You don't
need write permission on a file to unlink it.  Only on the directory.

Re: Dovecot correct ownership for logs

[Richard]

So, I've just written to the Dovecot mailing list, the reality why Dovecot
is complaining is so much worse than anything I could have imagined. While
everything indicates Dovecot is able to write to the log files, it seems
Dovecot expects execution permissions on the directory it writes the logs
to. Because "Standard POSIX permissions for a non-root process to enter a
directory." How on earth is that even a thing? Beyond binaries there
shouldn't really be any excuse for that many things to have execution
permissions on anything.

Re: Dovecot correct ownership for logs

[Richard]

>
> Don't call me a liar, you are just too dumb to understand.

It's sad to see that you need to make it this blatantly obvious that even I
clearly understand more than you do. And you're the one trying to scold me
about sticking to the mailing list rules when you so obviously don't care
for them yourself.

No the point is, you are not setting a file path, you are configure dovecot
> to directly write to these files.


I'm configuring dovecot the way it wants to be configured if you want to
deviate from default settings. What doesn't want to get to your brain is
that dovecot isn't the issue. Without postfix interfering everything would
be fine. It's postfix causing issues, not dovecot. Very simple logic would
have told you that, but you are way too focused on hatred and on knowing
everything better without actually doing so. I'm sorry but if you don't
have any clue of what you are talking about, just don't reply. Also you may
want to refrain from further replies - or attacks in your case - before you
write something you can't take back.

Re: Dovecot correct ownership for logs

[Greg Wooledge]

On Fri, May 17, 2024 at 08:20:17AM -0400, Henning Follmann wrote:
> No the point is, you are not setting a file path, you are configure dovecot
> to directly write to these files.
> And dovecot is not just one process, there are multiple running as
> different users all trying t write into one file. Race conditions are
> to be expected. Because these options exist does not mean it is  a good
> decision to use them.

This is speculation.  When writing to log files, one normally opens the
file in append mode.  Multiple processes opening a single log file in
append mode, and using atomic write() calls, will not have any locking
or concurrency issues.  It'll just work.

Now, the question is whether Dovecot and Postfix both work this way.  I
don't use those two programs, so I don't know the answer to that.

It has become apparent that nobody on this mailing list has the intimate
technical knowledge of Dovecot and/or Postfix required to assist the OP
with setting up this nonstandard logging configuration.  I would suggest
that the OP should try on a Dovecot or Postfix support list instead.

Re: Dovecot correct ownership for logs

[Henning Follmann]

On Fri, May 17, 2024 at 10:12:03AM +0200, Richard wrote:
> So now that you don't know any further, you just start lying? Now that's
> rich.
> 
> > I told you where to look, which is more than you deserve after how you
> behave.
> 
> You didn't though.

Don't call me a liar, you are just too dumb to understand.

> 
> > Configure the literal industry standard syslog or journald to use a facility
> to your liking and the problem should resolve itself.
> The point is, Dovecot has an option to write certain types of logs to
> different files. While it's doing that great, postfix is upset about that
> capability. It shouldn't even try to access these files. So the issue is
> not being able to log to files, that's already solved, but postfix running
> crazy when using a very simple setting.
>

No the point is, you are not setting a file path, you are configure dovecot
to directly write to these files.
And dovecot is not just one process, there are multiple running as
different users all trying t write into one file. Race conditions are
to be expected. Because these options exist does not mean it is  a good
decision to use them.
And because you clearly don't understand what you are doing, do as being told.
Configure syslog or rsyslog to use that file path.


-H


-- 
Henning Follmann   | hfollm...@itcfollmann.com

Re: Dovecot correct ownership for logs

[Richard]

So now that you don't know any further, you just start lying? Now that's
rich.

> I told you where to look, which is more than you deserve after how you
behave.

You didn't though.

> Configure the literal industry standard syslog or journald to use a facility
to your liking and the problem should resolve itself.
The point is, Dovecot has an option to write certain types of logs to
different files. While it's doing that great, postfix is upset about that
capability. It shouldn't even try to access these files. So the issue is
not being able to log to files, that's already solved, but postfix running
crazy when using a very simple setting.

Am Do., 16. Mai 2024 um 18:55 Uhr schrieb Henning Follmann <
hfollm...@itcfollmann.com>:

> On Thu, May 16, 2024 at 01:00:19PM +0200, Richard wrote:
> > But why is postfix even holding a lock on it? And how do I prevent that?
> I
> > never asked it to.
> > At least, I don't think there should be a different process holding a
> lock
> > on it.
> >
>
> I told you where to look, which is more than you deserve after how you
> behave.
> Configure the literal industry standard syslog or journald to use a
> facility to your liking and the problem should resolve itself.
>
> -H
>
>
> --
> Henning Follmann   | hfollm...@itcfollmann.com
>
>

Re: Dovecot correct ownership for logs

[Henning Follmann]

On Thu, May 16, 2024 at 01:00:19PM +0200, Richard wrote:
> But why is postfix even holding a lock on it? And how do I prevent that? I
> never asked it to.
> At least, I don't think there should be a different process holding a lock
> on it.
> 

I told you where to look, which is more than you deserve after how you
behave. 
Configure the literal industry standard syslog or journald to use a
facility to your liking and the problem should resolve itself.

-H 


-- 
Henning Follmann   | hfollm...@itcfollmann.com

Re: Dovecot correct ownership for logs

[Richard]

But why is postfix even holding a lock on it? And how do I prevent that? I
never asked it to.
At least, I don't think there should be a different process holding a lock
on it.

Am Mi., 15. Mai 2024 um 18:45 Uhr schrieb Henning Follmann <
hfollm...@itcfollmann.com>:

> On Wed, May 15, 2024 at 12:23:35PM +0200, Richard wrote:
> >
> [...]
>
> > But that's still not that helpful for the main issue. Why on earth is
> > postfix throwing issues about the log files, even when they are
> > world-readable and -writable? It's not that dovecot doesn't log to them,
> > but it's also not the case that it's an error message that can just be
> > ignored, as it brings mail delivery to a halt.
> >
>
> Maybe because you write directly to the logfile. One process trying to
> write
> t it while a different already holds a lock to it.
>
> -H
>
> --
> Henning Follmann   | hfollm...@itcfollmann.com
>
>

Re: Dovecot correct ownership for logs

[Henning Follmann]

On Wed, May 15, 2024 at 12:23:35PM +0200, Richard wrote:
>
[...]
 
> But that's still not that helpful for the main issue. Why on earth is
> postfix throwing issues about the log files, even when they are
> world-readable and -writable? It's not that dovecot doesn't log to them,
> but it's also not the case that it's an error message that can just be
> ignored, as it brings mail delivery to a halt.
> 

Maybe because you write directly to the logfile. One process trying to write
t it while a different already holds a lock to it.

-H

-- 
Henning Follmann   | hfollm...@itcfollmann.com

Re: Dovecot correct ownership for logs

[jeremy ardley]

On 15/5/24 18:52, Richard wrote:

mailbox_transport isn't defined anywhere.


Am Mi., 15. Mai 2024 um 12:37 Uhr schrieb jeremy ardley 
:



On 15/5/24 18:23, Richard wrote:
> Interesting. That's not even configured in our main.cfg. We have
these
> concerning dovecot:
> smtpd_sasl_type = dovecot
> mailbox_command = /usr/lib/dovecot/deliver -d $USER

The sasl line is not relevant

The mailbox_command is unusual. It means whatever process actually
execute the mailbox_command runs as (some) postfix user to run the
deliver application. This may well cause permission issues.

The usual case is dovecot listens for commands on a unix socket or
maybe
an IP socket. In any case it has an entirely separate user ID from
postfix.

You may want to look at using the mailbox_transport option instead of
the mailbox_command option

mailbox_transport =  lmtp:unix:private/dovecot-lmtp



Then you may want to look at the manuals and find out how to add a 
mailbox_transport entry and comment out the mailbox_command entry.


There are many other options of course, but mailbox_transport is a very 
common configuration and usually avoids most permission issues.

Re: Dovecot correct ownership for logs

[Richard]

mailbox_transport isn't defined anywhere.


Am Mi., 15. Mai 2024 um 12:37 Uhr schrieb jeremy ardley <
jeremy.ard...@gmail.com>:

>
> On 15/5/24 18:23, Richard wrote:
> > Interesting. That's not even configured in our main.cfg. We have these
> > concerning dovecot:
> > smtpd_sasl_type = dovecot
> > mailbox_command = /usr/lib/dovecot/deliver -d $USER
>
> The sasl line is not relevant
>
> The mailbox_command is unusual. It means whatever process actually
> execute the mailbox_command runs as (some) postfix user to run the
> deliver application. This may well cause permission issues.
>
> The usual case is dovecot listens for commands on a unix socket or maybe
> an IP socket. In any case it has an entirely separate user ID from postfix.
>
> You may want to look at using the mailbox_transport option instead of
> the mailbox_command option
>
> mailbox_transport  =  lmtp:unix:private/dovecot-lmtp
>
>

Re: Dovecot correct ownership for logs

[jeremy ardley]

On 15/5/24 18:23, Richard wrote:
Interesting. That's not even configured in our main.cfg. We have these 
concerning dovecot:

smtpd_sasl_type = dovecot
mailbox_command = /usr/lib/dovecot/deliver -d $USER


The sasl line is not relevant

The mailbox_command is unusual. It means whatever process actually 
execute the mailbox_command runs as (some) postfix user to run the 
deliver application. This may well cause permission issues.


The usual case is dovecot listens for commands on a unix socket or maybe 
an IP socket. In any case it has an entirely separate user ID from postfix.


You may want to look at using the mailbox_transport option instead of 
the mailbox_command option


mailbox_transport  =  lmtp:unix:private/dovecot-lmtp

Re: Dovecot correct ownership for logs

[Richard]

Interesting. That's not even configured in our main.cfg. We have these
concerning dovecot:
smtpd_sasl_type = dovecot
mailbox_command = /usr/lib/dovecot/deliver -d $USER

But that's still not that helpful for the main issue. Why on earth is
postfix throwing issues about the log files, even when they are
world-readable and -writable? It's not that dovecot doesn't log to them,
but it's also not the case that it's an error message that can just be
ignored, as it brings mail delivery to a halt.

Am Mi., 15. Mai 2024 um 05:00 Uhr schrieb jeremy ardley <
jeremy.ard...@gmail.com>:


> This can be configured in postfix main.conf as
>
> virtual_transport  =  lmtp:unix:private/dovecot-lmtp or mailbox_transport
> =  lmtp:unix:private/dovecot-lmtp
>
>  From the postfix howto guide
>
> mailbox_transport_maps (default: empty)
>
>  Optional lookup tables with per-recipient message delivery transports
> to use for local(8) mailbox delivery, whether or not the recipients are
> found in the UNIX passwd database.
>
>  The precedence of local(8) delivery features from high to low is:
> aliases, .forward files, mailbox_transport_maps, mailbox_transport,
> mailbox_command_maps,
>  mailbox_command, home_mailbox, mail_spool_directory,
> fallback_transport_maps, fallback_transport and luser_relay.
>
> https://www.postfix.org/local.8.html
>
> https://doc.dovecot.org/configuration_manual/howto/postfix_dovecot_lmtp/
>
>

Re: Dovecot correct ownership for logs

[jeremy ardley]

On 14/5/24 20:17, to...@tuxteam.de wrote:

On Tue, May 14, 2024 at 02:11:53PM +0200, Richard wrote:

[...]


Setting the permissions in /var/log/dovecot to 666 actually didn't
solve the problem [...]

This seems to prove (or, at least, strongly suggest) that I was barking
up the wrong tree. I've currently run out of trees and at $DAYJOB, so
tight on resources. Good luck :)


Clarifying my understanding of the issues I have discovered that postfix runs a 
non chroot service 'local' that has the initial responsibility to deliver mail 
locally.

local runs as root and has the ability to deliver mail to local files

local also has the ability to delegate the delivery to dovecot and other 
agents. This can be configured in postfix main.conf as

virtual_transport  =  lmtp:unix:private/dovecot-lmtp or mailbox_transport  =  
lmtp:unix:private/dovecot-lmtp

From the postfix howto guide

mailbox_transport_maps (default: empty)

    Optional lookup tables with per-recipient message delivery transports to 
use for local(8) mailbox delivery, whether or not the recipients are found in 
the UNIX passwd database.

    The precedence of local(8) delivery features from high to low is: aliases, 
.forward files, mailbox_transport_maps, mailbox_transport, mailbox_command_maps,
mailbox_command, home_mailbox, mail_spool_directory, 
fallback_transport_maps, fallback_transport and luser_relay.

https://www.postfix.org/local.8.html

https://doc.dovecot.org/configuration_manual/howto/postfix_dovecot_lmtp/

Re: Dovecot correct ownership for logs

[Richard]

Says the one refusing to stay on topic. What a sad hypocrite.

On Tue, May 14, 2024, 20:10 Henning Follmann 
wrote:

> On Tue, May 14, 2024 at 03:11:16PM +0200, Richard wrote:
> > "Top posting" (writing the answer above the text that's being replied to)
> > is literally industry standard behavior.
> >
> Whatever.
> It is not standard behavior in mailing lists.
>
> https://wiki.debian.org/DebianMailingLists#Posting_Rules.2C_Guidelines.2C_and_Tips
>
> But I just asked you to not to. You can ignore that, as you do.
>
> Good luck with your issues.
>
> > Also, I don't think you've really cleared out any confusion. Now, how
> > exactly can dovecot log to /var/log/dovecot/ without (postfix) throwing
> > errors? Because it clearly is for 2 out of 3 files as visible from the
> file
> > sizes in my original post. Only the debug file is empty, but maybe
> nothing
> > relevant enough has been found yet. There are entries from master,
> > imap, imap-login, managesieve-login, anvil and various other processes in
> > info and error. But the error messages from postfix still appear, also
> > seemingly at least vastly slowing down emails being delivered to the
> users,
> > if not bringing that to a complete halt. Only after commenting out these
> > locations in 10-logging.conf the mails show up for the users.
> >
>
>
> -H
>
>
> --
> Henning Follmann   | hfollm...@itcfollmann.com
>
>

Re: Dovecot correct ownership for logs

[Richard]

How exactly do I do that?

Am Di., 14. Mai 2024 um 18:40 Uhr schrieb jeremy ardley <
jeremy.ard...@gmail.com>:

>
>  From what I can find out, the postfix local delivery agent is not
> chroot and it communicates with the main postfix processes via shared
> directories and pipes.
>
> To debug the problem I suggest checking the chroot and non chroot
> processes and eliminate those that are not relevant to the seen error
> messages
>
>

Re: Dovecot correct ownership for logs

[Nicolas George]

Alain D D Williams (12024-05-14):
> PS: check the dictionary definition of "literally".

I think you should have checked first that it makes the point you want
to make and not the opposite:

2. (degree, figuratively, proscribed, contranym) Used non-literally as
   an intensifier for figurative statements: virtually, so to speak
   (often considered incorrect; see usage notes)
3. (colloquial) Used to intensify or dramatize non-figurative
   statements.
4. (colloquial) Used as a generic downtoner: just, merely.


2. in effect : virtually—used in an exaggerated way to emphasize a
   statement or description that is not literally true or possible


* used to emphasize what you are saying:
* simply or just:


Regards,

-- 
  Nicolas George

Re: Dovecot correct ownership for logs

[Alain D D Williams]

On Tue, May 14, 2024 at 03:11:16PM +0200, Richard wrote:
>"Top posting" (writing the answer above the text that's being replied
>to) is literally industry standard behavior.

Many do top post, but many do not.

Places where it is often frowned on are technical mail lists such as this one.
This is because only quoting to the parts of the mail that you reply to and
putting you comment underneath can greatly help understanding.

Read the Netiquette Guidelines (1995):

https://www.ietf.org/rfc/rfc1855.txt

Other discussions here:

https://idallen.com/topposting.html

https://www.caliburn.nl/topposting.html

PS: check the dictionary definition of "literally".

-- 
Alain Williams
Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT 
Lecturer.
+44 (0) 787 668 0256  https://www.phcomp.co.uk/
Parliament Hill Computers. Registration Information: 
https://www.phcomp.co.uk/Contact.html
#include 

Re: OT: Top Posting (was: Dovecot correct ownership for logs)

[tomas]

On Tue, May 14, 2024 at 04:08:19PM +0200, Richard wrote:
> Just because something isn't an official ISO standard doesn't mean it's not
> standard behavior. And how it relates to this mailing list? It's called a
> setting.

Most people prefer inline quoting around here (I know I do). That's
because for big mailing lists, with long threads, it works much, much
better.

That said, we usually are tolerant of top posts. What gets me
is the hostility of your reaction. You aren't going to convince
anyone. Even not with "industry standards" [1]

As far as your main concern goes... I lost interest.

Cheers

[1] Q: How many Microsoft technicians does it take to change a
   light bulb?
A: None, they just redefine Darkness (TM) as the new industry
   standard.

https://www.linux.com/news/how-many-microsoft-technicians-does-it-take-change-light-bulb/

-- 
t


signature.asc
Description: PGP signature

Re: OT: Top Posting (was: Dovecot correct ownership for logs)

[Richard]

Just because something isn't an official ISO standard doesn't mean it's not
standard behavior. And how it relates to this mailing list? It's called a
setting.

Am Di., 14. Mai 2024 um 15:57 Uhr schrieb Loris Bennett <
loris.benn...@fu-berlin.de>:

> Hi Richard,
>
> Richard  writes:
>
> > "Top posting" (writing the answer above the text that's being replied
> > to) is literally industry standard behavior.
>
> Can you provide a link to the standard you are referring to?
>
> Assuming such a standard exists, how would it apply to this newsgroup?
>
> [snip (51 lines)]
>
> Cheers,
>
> Loris
>
> --
> This signature is currently under constuction.
>
>

OT: Top Posting (was: Dovecot correct ownership for logs)

[Loris Bennett]

Hi Richard,

Richard  writes:

> "Top posting" (writing the answer above the text that's being replied
> to) is literally industry standard behavior.

Can you provide a link to the standard you are referring to?

Assuming such a standard exists, how would it apply to this newsgroup?

[snip (51 lines)]

Cheers,

Loris

-- 
This signature is currently under constuction.

Re: Dovecot correct ownership for logs

[Henning Follmann]

On Tue, May 14, 2024 at 03:11:16PM +0200, Richard wrote:
> "Top posting" (writing the answer above the text that's being replied to)
> is literally industry standard behavior.
> 
Whatever.
It is not standard behavior in mailing lists.
https://wiki.debian.org/DebianMailingLists#Posting_Rules.2C_Guidelines.2C_and_Tips

But I just asked you to not to. You can ignore that, as you do.

Good luck with your issues.

> Also, I don't think you've really cleared out any confusion. Now, how
> exactly can dovecot log to /var/log/dovecot/ without (postfix) throwing
> errors? Because it clearly is for 2 out of 3 files as visible from the file
> sizes in my original post. Only the debug file is empty, but maybe nothing
> relevant enough has been found yet. There are entries from master,
> imap, imap-login, managesieve-login, anvil and various other processes in
> info and error. But the error messages from postfix still appear, also
> seemingly at least vastly slowing down emails being delivered to the users,
> if not bringing that to a complete halt. Only after commenting out these
> locations in 10-logging.conf the mails show up for the users.
> 


-H


-- 
Henning Follmann   | hfollm...@itcfollmann.com

Re: Dovecot correct ownership for logs

[Richard]

And you think you're important enough to change that setting for a whole
mailing account? You're funny.

Am Di., 14. Mai 2024 um 15:16 Uhr schrieb Brad Rogers :

> On Tue, 14 May 2024 15:11:16 +0200
> Richard  wrote:
>
> Hello Richard,
>
> >"Top posting" (writing the answer above the text that's being replied
> >to) is literally industry standard behavior.
>
> This 'literally' isn't industry.
>
> --
>  Regards  _   "Valid sig separator is {dash}{dash}{space}"
>  / )  "The blindingly obvious is never immediately apparent"
> / _)rad   "Is it only me that has a working delete key?"
> Is she really going out with him?
> New Rose - The Damned
>

Re: Dovecot correct ownership for logs

[Brad Rogers]

On Tue, 14 May 2024 15:11:16 +0200
Richard  wrote:

Hello Richard,

>"Top posting" (writing the answer above the text that's being replied
>to) is literally industry standard behavior.

This 'literally' isn't industry.

-- 
 Regards  _   "Valid sig separator is {dash}{dash}{space}"
 / )  "The blindingly obvious is never immediately apparent"
/ _)rad   "Is it only me that has a working delete key?"
Is she really going out with him?
New Rose - The Damned


pgp8KT3gXNAal.pgp
Description: OpenPGP digital signature

Re: Dovecot correct ownership for logs

[Richard]

"Top posting" (writing the answer above the text that's being replied to)
is literally industry standard behavior.

Also, I don't think you've really cleared out any confusion. Now, how
exactly can dovecot log to /var/log/dovecot/ without (postfix) throwing
errors? Because it clearly is for 2 out of 3 files as visible from the file
sizes in my original post. Only the debug file is empty, but maybe nothing
relevant enough has been found yet. There are entries from master,
imap, imap-login, managesieve-login, anvil and various other processes in
info and error. But the error messages from postfix still appear, also
seemingly at least vastly slowing down emails being delivered to the users,
if not bringing that to a complete halt. Only after commenting out these
locations in 10-logging.conf the mails show up for the users.

Am Di., 14. Mai 2024 um 14:54 Uhr schrieb Henning Follmann <
hfollm...@itcfollmann.com>:

> On Tue, May 14, 2024 at 01:58:07PM +0200, Richard wrote:
> >
>
> Please don't top post! It is a pain to follow a thread if you do.
>
>
> > Am Di., 14. Mai 2024 um 13:45 Uhr schrieb Greg Wooledge <
> g...@wooledge.org>:
> >
> > > On Tue, May 14, 2024 at 07:36:17PM +0800, jeremy ardley wrote:
> > > > Postfix is chrooted (usuallly) to /var/spool/postfix
> > >
> > > If this is true, then how would a local delivery agent work?  It needs
> > > write access to all users' inboxes, which are either in /var/mail or in
> > > users' home directories.
>
> Usually the local delivery is done by a local agent (procmail, dovecot's
> lmtp daemon). These do not run chrooted.
>
> > >
> > > I could imagine the Postfix SMTP sending/receiving and queue processing
> > > programs being chrooted, but the LDA probably isn't.  Or at least not
> > > chrooted to /var/spool/postfix.
> > >
> > >
> [fixed top posting]
> > For us the situation is even a bit stranger. The inboxes are located in
> > neither location, but in /maildirs/username/ (no idea why it was set up
> > that way, but it's a dedicated mail server where the user's don't have
> > their own home directory). /var/mail is empty.
>
> This is a common dovecot setup.
> lmtp provides a unix socket into postfix chroot environment. This process
> usually runs as postfix user so the socket is owned by postfix. But it uses
> the dovecot logging configuration.
> Having the maildirs in a dedicated directory makes it easy to have a
> dedicated drive just for the mailboxes. This makes adapting to growth much
> simpler.
>
> -H
>
>
> --
> Henning Follmann   | hfollm...@itcfollmann.com
>
>

Re: Dovecot correct ownership for logs

[tomas]

On Tue, May 14, 2024 at 02:11:53PM +0200, Richard wrote:

[...]

> Setting the permissions in /var/log/dovecot to 666 actually didn't
> solve the problem [...]

This seems to prove (or, at least, strongly suggest) that I was barking
up the wrong tree. I've currently run out of trees and at $DAYJOB, so
tight on resources. Good luck :)

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Dovecot correct ownership for logs

[Richard]

>
> ps -eo pid,user,group,comm | grep postfix
> 2886706 postfix  postfix  pickup
> 2886707 postfix  postfix  qmgr
> 2886764 postfix  postfix  tlsmgr

Also as far as I know, postfix logs to syslog too. At least there is no
dedicated file or folder for it in /var/log.

Setting the permissions in /var/log/dovecot to 666 actually didn't
solve the problem, which just opens a whole other bunch of questions. So in
case that for some odd reason AppArmor logs aren't logged to syslog (and
also it doesn't have a dedicated file), these are the rules for dovecot and
postfix I could find:
postfix has an apparmor (in abstractions) file that doesn't say anything
about /var/log. It only has these rules for things in /var:

/var/spool/postfix/etc/*r,
/var/spool/postfix/lib/lib*.so* mr,
/var/spool/postfix/lib/@{multiarch}/lib*.so* mr,

Dovecot has two files. In tunables you can find this:
# @{DOVECOT_MAILSTORE} is a space-separated list of all directories
# where dovecot is allowed to store and read mails
#
# The default value is quite broad to avoid breaking existing setups.
# Please change @{DOVECOT_MAILSTORE} to (only) contain the directory
# you use, and remove everything else.

@{DOVECOT_MAILSTORE}=@{HOME}/Maildir/ @{HOME}/mail/ @{HOME}/Mail/
/var/vmail/ /var/mail/ /var/spool/mail

Which doesn't seem to be relevant for this. No idea how dovecot can put the
mail into /maildirs/username, but since that's working I'm not complaining.
The file in abstractions only contains this:
# used with dovecot/*

  abi ,

  capability setgid,

  deny capability block_suspend,

  # dovecot's master can send us signals
  signal receive peer=dovecot,

  owner @{run}/dovecot/config rw,

  # Include additions to the abstraction
  include if exists 

Am Di., 14. Mai 2024 um 13:45 Uhr schrieb :

> On Tue, May 14, 2024 at 01:29:17PM +0200, Richard wrote:
> > My guess is that postfix runs as postfix.
>
> That would be my guess too (or perhaps as some special "Debian-+postfix".
>
> > At least processes like local,
> > smtpd, bounce etc run as that user. But beyond that I have no idea how to
> > find that out. At least there's nothing in the postfix.service or
> > postfix@.service
> > about that. So I've changed the files to dovecot:postfix 664, but same
> > error.
>
> You might try
>
>   ps -eo pid,user,group,comm | grep postfix
>
> or similar. Or have a look at Posrfix's log file ownerships.
>
> You might try making the log files in question world writable just
> to see whether the problem disappears or this approach is a blind
> alley (don't forget to revert that: leaving them world-writable
> seems like asking for trouble).
>
> Cheers
> --
> t
>

Re: Dovecot correct ownership for logs

[Florent Rougon]

Le 14/05/2024, to...@tuxteam.de a écrit:

> You might try
>
>   ps -eo pid,user,group,comm | grep postfix
>
> or similar.

Yep, and beware that the original message mentions a postfix program
named 'local' (/usr/lib/postfix/sbin/local).

> May 13 20:55:37 mail postfix/local[2824184]: (...)

Regards

-- 
Florent

Re: Dovecot correct ownership for logs

[Richard]

For us the situation is even a bit stranger. The inboxes are located in
neither location, but in /maildirs/username/ (no idea why it was set up
that way, but it's a dedicated mail server where the user's don't have
their own home directory). /var/mail is empty.

Am Di., 14. Mai 2024 um 13:45 Uhr schrieb Greg Wooledge :

> On Tue, May 14, 2024 at 07:36:17PM +0800, jeremy ardley wrote:
> > Postfix is chrooted (usuallly) to /var/spool/postfix
>
> If this is true, then how would a local delivery agent work?  It needs
> write access to all users' inboxes, which are either in /var/mail or in
> users' home directories.
>
> I could imagine the Postfix SMTP sending/receiving and queue processing
> programs being chrooted, but the LDA probably isn't.  Or at least not
> chrooted to /var/spool/postfix.
>
>

Re: Dovecot correct ownership for logs

[jeremy ardley]

On 14/5/24 19:44, Greg Wooledge wrote:

On Tue, May 14, 2024 at 07:36:17PM +0800, jeremy ardley wrote:

Postfix is chrooted (usuallly) to /var/spool/postfix

If this is true, then how would a local delivery agent work?  It needs
write access to all users' inboxes, which are either in /var/mail or in
users' home directories.

I could imagine the Postfix SMTP sending/receiving and queue processing
programs being chrooted, but the LDA probably isn't.  Or at least not
chrooted to /var/spool/postfix.

From what I can find out, the postfix local delivery agent is not 
chroot and it communicates with the main postfix processes via shared 
directories and pipes.


To debug the problem I suggest checking the chroot and non chroot 
processes and eliminate those that are not relevant to the seen error 
messages

Re: Dovecot correct ownership for logs

[tomas]

On Tue, May 14, 2024 at 07:36:17PM +0800, jeremy ardley wrote:

[...]

> Postfix is chrooted (usuallly) to /var/spool/postfix
> 
> If postfix complains about /var/log/dovecot it's actually complaining about
> /var/spool/postfix/var/log/dovecot

I'm sceptical about this -- the error would have been ENOENT, not EPERM
(because an intervening directory would be missing).

But of course, I might be wrong.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Dovecot correct ownership for logs

[tomas]

On Tue, May 14, 2024 at 01:29:17PM +0200, Richard wrote:
> My guess is that postfix runs as postfix.

That would be my guess too (or perhaps as some special "Debian-+postfix".

> At least processes like local,
> smtpd, bounce etc run as that user. But beyond that I have no idea how to
> find that out. At least there's nothing in the postfix.service or
> postfix@.service
> about that. So I've changed the files to dovecot:postfix 664, but same
> error.

You might try

  ps -eo pid,user,group,comm | grep postfix

or similar. Or have a look at Posrfix's log file ownerships.

You might try making the log files in question world writable just
to see whether the problem disappears or this approach is a blind
alley (don't forget to revert that: leaving them world-writable
seems like asking for trouble).

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Dovecot correct ownership for logs

[Greg Wooledge]

On Tue, May 14, 2024 at 07:36:17PM +0800, jeremy ardley wrote:
> Postfix is chrooted (usuallly) to /var/spool/postfix

If this is true, then how would a local delivery agent work?  It needs
write access to all users' inboxes, which are either in /var/mail or in
users' home directories.

I could imagine the Postfix SMTP sending/receiving and queue processing
programs being chrooted, but the LDA probably isn't.  Or at least not
chrooted to /var/spool/postfix.

Re: Dovecot correct ownership for logs

[jeremy ardley]

On 14/5/24 19:17, Richard wrote:
But why should it cause issues? I set the logging in dovecot's conf.d, 
so I'd expect dovecot to write these logs, not postfix as it has its 
own settings.


Am Di., 14. Mai 2024 um 05:00 Uhr schrieb jeremy ardley 
:



On 14/5/24 04:16, Richard wrote:
> Maybe someone here knows how the ownership of these files for
Dovecot
> needs to be in order to work, as various distributions of Dovecot
> packages seem to use different users:
> I'd like Dovecot not to log into syslog, but to dedicated files.
> Therefore I've created the directory /var/log/dovecot and told
dovecot
> in 10-logging.conf to log info, debug and error messages to
separate
> files.  But I get error messages from postfix (weird):


A possible cause of problems between postfix and dovecot is that
postfix
usuallly runs chroot so the file tree it sees is not the same as
the one
the rest of the system sees.



Postfix is chrooted (usuallly) to /var/spool/postfix

If postfix complains about /var/log/dovecot it's actually complaining 
about /var/spool/postfix/var/log/dovecot


I would check that file/directory and how dovecot and postfix interact 
with it.

Re: Dovecot correct ownership for logs

[Richard]

My guess is that postfix runs as postfix. At least processes like local,
smtpd, bounce etc run as that user. But beyond that I have no idea how to
find that out. At least there's nothing in the postfix.service or
postfix@.service
about that. So I've changed the files to dovecot:postfix 664, but same
error.

Am Di., 14. Mai 2024 um 06:34 Uhr schrieb :

> On Mon, May 13, 2024 at 10:16:13PM +0200, Richard wrote:
> > Maybe someone here knows how the ownership of these files for Dovecot
> needs
> > to be in order to work, as various distributions of Dovecot packages seem
> > to use different users:
> > I'd like Dovecot not to log into syslog, but to dedicated files.
> Therefore
> > I've created the directory /var/log/dovecot and told dovecot in
> > 10-logging.conf to log info, debug and error messages to separate files.
> > But I get error messages from postfix (weird):
>
> I think this Dovecot's LDA (the local delivery agent) [1], which is
> invoked by the MTA (Postfix) and is, therefore, most probably running
> as postfix.
>
> [...]
>
> > > (temporary failure. Command output: lda(user): Error:
> > > net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied
> Can't
> > > open log file /var/log/dovecot/error.log: Permission denied )
>
> This message actually is an indicator against the chroot theory posed
> elsewhere in this thread (in a chroot, you would get "no such file or
> directory", I guess).
> >
> > This is the content of /var/log/dovecot:
> > -rw-r--r--  1 dovecot dovecot0 13. Mai 20:50 debug.log
> > -rw-r--r--  1 dovecot dovecot  880 13. Mai 21:21 error.log
> > -rw-r--r--  1 dovecot dovecot  40K 13. Mai 21:20 info.log
>
> Try to set the log file's group to mail (or whatever group Postfix is
> running as) and make them group writable.
>
> Cheers
> --
> t
>

Re: Dovecot correct ownership for logs

[Richard]

AppArmor complaints would be shown in journalctl too. But dmseg doesn't
show anything either. Just switched dovecot back to these log files, waited
for the error message, yet dmesg doesn't have anything new since yesterday.
Systemd was also my guess as it was originally set to ProtectSystem=full
for dovecot.service, but reducing that to yes didn't change anything.

These are the in the Service block of the service, maybe you see anything
that could be a reason too:
[Service]
Type=notify
ExecStart=/usr/sbin/dovecot -F
ExecReload=/usr/bin/doveadm reload
ExecStop=/usr/bin/doveadm stop
PrivateTmp=true
NonBlocking=yes
ProtectSystem=yes
ProtectHome=no
PrivateDevices=true
Restart=on-failure

Best
Richard

Am Mo., 13. Mai 2024 um 22:28 Uhr schrieb Greg Wooledge :

> On Mon, May 13, 2024 at 10:16:13PM +0200, Richard wrote:
> > May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: to=<
> u...@domain.de>,
> > > relay=local, delay=3.2, delays=1.9/0.29/0/1.1, dsn=4.3.0,
> status=deferred
> > > (temporary failure. Command output: lda(user): Error:
> > > net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied
> Can't
> > > open log file /var/log/dovecot/error.log: Permission denied )
> >
> > This is the content of /var/log/dovecot:
> > -rw-r--r--  1 dovecot dovecot0 13. Mai 20:50 debug.log
> > -rw-r--r--  1 dovecot dovecot  880 13. Mai 21:21 error.log
> > -rw-r--r--  1 dovecot dovecot  40K 13. Mai 21:20 info.log
>
> The first two things that come to mind are AppArmor, and systemd unit
> files.
>
> Check to see whether there's an AppArmor profile for this program, or
> any lines from AppArmor in dmesg.
>
> If that's not it, look at the systemd unit file(s) that start the
> dovecot service(s), if there are any, and see if they're using any of
> the directives that restrict the locations the program can access.
>
>

Re: Dovecot correct ownership for logs

[Richard]

But why should it cause issues? I set the logging in dovecot's conf.d, so
I'd expect dovecot to write these logs, not postfix as it has its own
settings.

Am Di., 14. Mai 2024 um 05:00 Uhr schrieb jeremy ardley <
jeremy.ard...@gmail.com>:

>
> On 14/5/24 04:16, Richard wrote:
> > Maybe someone here knows how the ownership of these files for Dovecot
> > needs to be in order to work, as various distributions of Dovecot
> > packages seem to use different users:
> > I'd like Dovecot not to log into syslog, but to dedicated files.
> > Therefore I've created the directory /var/log/dovecot and told dovecot
> > in 10-logging.conf to log info, debug and error messages to separate
> > files.  But I get error messages from postfix (weird):
>
>
> A possible cause of problems between postfix and dovecot is that postfix
> usuallly runs chroot so the file tree it sees is not the same as the one
> the rest of the system sees.
>
>

Re: Dovecot correct ownership for logs

[tomas]

On Mon, May 13, 2024 at 10:16:13PM +0200, Richard wrote:
> Maybe someone here knows how the ownership of these files for Dovecot needs
> to be in order to work, as various distributions of Dovecot packages seem
> to use different users:
> I'd like Dovecot not to log into syslog, but to dedicated files. Therefore
> I've created the directory /var/log/dovecot and told dovecot in
> 10-logging.conf to log info, debug and error messages to separate files.
> But I get error messages from postfix (weird):

I think this Dovecot's LDA (the local delivery agent) [1], which is
invoked by the MTA (Postfix) and is, therefore, most probably running
as postfix.

[...]

> > (temporary failure. Command output: lda(user): Error:
> > net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied Can't
> > open log file /var/log/dovecot/error.log: Permission denied )

This message actually is an indicator against the chroot theory posed
elsewhere in this thread (in a chroot, you would get "no such file or
directory", I guess).
> 
> This is the content of /var/log/dovecot:
> -rw-r--r--  1 dovecot dovecot0 13. Mai 20:50 debug.log
> -rw-r--r--  1 dovecot dovecot  880 13. Mai 21:21 error.log
> -rw-r--r--  1 dovecot dovecot  40K 13. Mai 21:20 info.log

Try to set the log file's group to mail (or whatever group Postfix is
running as) and make them group writable.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Dovecot correct ownership for logs

[jeremy ardley]

On 14/5/24 04:16, Richard wrote:
Maybe someone here knows how the ownership of these files for Dovecot 
needs to be in order to work, as various distributions of Dovecot 
packages seem to use different users:
I'd like Dovecot not to log into syslog, but to dedicated files. 
Therefore I've created the directory /var/log/dovecot and told dovecot 
in 10-logging.conf to log info, debug and error messages to separate 
files.  But I get error messages from postfix (weird):



A possible cause of problems between postfix and dovecot is that postfix 
usuallly runs chroot so the file tree it sees is not the same as the one 
the rest of the system sees.

Re: Dovecot correct ownership for logs

[Geert Stappers]

On Mon, May 13, 2024 at 10:16:13PM +0200, Richard wrote:
> Maybe someone here knows how the ownership of these files for Dovecot needs
> to be in order to work, as various distributions of Dovecot packages seem
> to use different users:
> I'd like Dovecot not to log into syslog, but to dedicated files. Therefore
> I've created the directory /var/log/dovecot and told dovecot in
> 10-logging.conf to log info, debug and error messages to separate files.
> But I get error messages from postfix (weird):
> 
> May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: to=,
> > relay=local, delay=3.2, delays=1.9/0.29/0/1.1, dsn=4.3.0, status=deferred
> > (temporary failure. Command output: lda(user): Error:
> > net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied Can't
> > open log file /var/log/dovecot/error.log: Permission denied )
> 
> This is the content of /var/log/dovecot:
> -rw-r--r--  1 dovecot dovecot0 13. Mai 20:50 debug.log
> -rw-r--r--  1 dovecot dovecot  880 13. Mai 21:21 error.log
> -rw-r--r--  1 dovecot dovecot  40K 13. Mai 21:20 info.log
> 
> So why is Dovecot complaining?

:-)


> I only see processes owned by root, dovecot,
> dovenull and user processes of dovecot. And dovenull seems to be not
> relevant for this.


Reread
} May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: to=,
} } open log file /var/log/dovecot/error.log: Permission denied )

Note  postfix

 
> Best
> Richard


Groeten
Geert Stappers
-- 
Silence is hard to parse

Re: Dovecot correct ownership for logs

[Greg Wooledge]

On Mon, May 13, 2024 at 10:16:13PM +0200, Richard wrote:
> May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: to=,
> > relay=local, delay=3.2, delays=1.9/0.29/0/1.1, dsn=4.3.0, status=deferred
> > (temporary failure. Command output: lda(user): Error:
> > net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied Can't
> > open log file /var/log/dovecot/error.log: Permission denied )
> 
> This is the content of /var/log/dovecot:
> -rw-r--r--  1 dovecot dovecot0 13. Mai 20:50 debug.log
> -rw-r--r--  1 dovecot dovecot  880 13. Mai 21:21 error.log
> -rw-r--r--  1 dovecot dovecot  40K 13. Mai 21:20 info.log

The first two things that come to mind are AppArmor, and systemd unit
files.

Check to see whether there's an AppArmor profile for this program, or
any lines from AppArmor in dmesg.

If that's not it, look at the systemd unit file(s) that start the
dovecot service(s), if there are any, and see if they're using any of
the directives that restrict the locations the program can access.

Dovecot correct ownership for logs

[Richard]

Maybe someone here knows how the ownership of these files for Dovecot needs
to be in order to work, as various distributions of Dovecot packages seem
to use different users:
I'd like Dovecot not to log into syslog, but to dedicated files. Therefore
I've created the directory /var/log/dovecot and told dovecot in
10-logging.conf to log info, debug and error messages to separate files.
But I get error messages from postfix (weird):

May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: to=,
> relay=local, delay=3.2, delays=1.9/0.29/0/1.1, dsn=4.3.0, status=deferred
> (temporary failure. Command output: lda(user): Error:
> net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied Can't
> open log file /var/log/dovecot/error.log: Permission denied )

This is the content of /var/log/dovecot:
-rw-r--r--  1 dovecot dovecot0 13. Mai 20:50 debug.log
-rw-r--r--  1 dovecot dovecot  880 13. Mai 21:21 error.log
-rw-r--r--  1 dovecot dovecot  40K 13. Mai 21:20 info.log

So why is Dovecot complaining? I only see processes owned by root, dovecot,
dovenull and user processes of dovecot. And dovenull seems to be not
relevant for this.

Best
Richard