Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

David Wright wrote:

> FYI:
>
> I installed iwatch, and that immediately generated two messages from
> /etc/.etckeeper. Then I upgraded:
>
>   apt apt-doc apt-utils bind9-host curl dnsutils exim4
> exim4-base exim4-config exim4-daemon-light
>   firefox-esr firefox-esr-l10n-en-gb gstreamer1.0-gl
> gstreamer1.0-libav gstreamer1.0-plugins-bad
>   gstreamer1.0-plugins-base gstreamer1.0-plugins-good
> gstreamer1.0-pulseaudio gstreamer1.0-x
>   libapt-inst2.0 libapt-pkg5.0 libbind9-161 libcurl3-gnutls
> libcurl4 libdns-export1104 libdns1104
>   libgstreamer-gl1.0-0 libgstreamer-plugins-bad1.0-0
> libgstreamer-plugins-base1.0-0 libirs161
>   libisc-export1100 libisc1100 libisccc161 libisccfg163
> libjs-underscore libldb1 liblwres161
>   libopenjp2-7 openjdk-11-jre openjdk-11-jre-headless
> wpasupplicant xserver-common
>   xserver-xorg-core xserver-xorg-legacy
> 44 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>
> and got 387 more messages.
>
> I then added one new user, which generated 97 more, where
> /etc/.pwd.lock was the subject of four of them.
>
> Purging iwatch then generated a final three.
>
> So the OP's 2757 is no surprise with the
> default configuration.

Really, I was 100% it was something _I_ did...

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

Jonathan Dowland wrote:

> On Wed, May 05, 2021 at 03:51:16PM +0200, Emanuel Berg wrote:
>
>> I learned something as well, how to delete mails. First see
>> how many mails there are, say there are 756, then type
>> t 1-756 RET and then hold down q :)
>
> That's a slow way. "T iwatch ; d" or "T ~b iwatch ; d" would
> be faster (using ; bound to tag-prefix: apply next function
> to tagged messages)

d * even faster perhaps...

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[David Wright]

On Wed 05 May 2021 at 07:26:34 (-0400), Greg Wooledge wrote:
> On Tue, May 04, 2021 at 09:32:49PM -0500, David Wright wrote:
> > It looks reasonable for determining whether your system files are
> > being interfered with. But you just showed one example from the
> > log, which was for the /etc/.pwd.lock lockfile. I assume you don't
> > have 2757 of these but, rather, the names of an assortment of files.
> 
> That's an interesting interpretation.  If that's actually *true*, I
> wish the OP had made that more clear.  I interpreted it as literally
> being thousands of instances of the *same* file, the one shown in the
> Subject: header and in the original message body.
> 
> (In which case, removing iwatch will certainly stop the logging, but
> it won't stop whoever is locking and unlocking your passwd/shadow
> files thousands of times, which is something I might care enough to
> investigate -- and is a great reason for installing iwatch, to look for
> such a thing.)
> 
> (Also I'd never heard of "monkeysphere" before and didn't even know
> that openssh-client suggested it.  So it's been an educational thread.)

FYI:

I installed iwatch, and that immediately generated two messages from
/etc/.etckeeper. Then I upgraded:

  apt apt-doc apt-utils bind9-host curl dnsutils exim4 exim4-base exim4-config 
exim4-daemon-light
  firefox-esr firefox-esr-l10n-en-gb gstreamer1.0-gl gstreamer1.0-libav 
gstreamer1.0-plugins-bad
  gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-pulseaudio 
gstreamer1.0-x
  libapt-inst2.0 libapt-pkg5.0 libbind9-161 libcurl3-gnutls libcurl4 
libdns-export1104 libdns1104
  libgstreamer-gl1.0-0 libgstreamer-plugins-bad1.0-0 
libgstreamer-plugins-base1.0-0 libirs161
  libisc-export1100 libisc1100 libisccc161 libisccfg163 libjs-underscore 
libldb1 liblwres161
  libopenjp2-7 openjdk-11-jre openjdk-11-jre-headless wpasupplicant 
xserver-common
  xserver-xorg-core xserver-xorg-legacy
44 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

and got 387 more messages.

I then added one new user, which generated 97 more, where
/etc/.pwd.lock was the subject of four of them.

Purging iwatch then generated a final three.

So the OP's 2757 is no surprise with the default configuration.

Cheers,
David.

Re: repeated system mail, /etc/.pwd.lock ?

[Jonathan Dowland]

On Wed, May 05, 2021 at 03:51:16PM +0200, Emanuel Berg wrote:

I learned something as well, how to delete mails. First see
how many mails there are, say there are 756, then type t 1-756
RET and then hold down q :)


That's a slow way. "T iwatch ; d" or "T ~b iwatch ; d" would be faster
(using ; bound to tag-prefix: apply next function to tagged messages)


--
Please do not CC me, I am subscribed to the list.

  Jonathan Dowland
✎j...@debian.org
   https://jmtd.net

Re: OT: dotfile generalities (was Re: repeated system mail, /etc/.pwd.lock ?)

[tomas]

On Fri, May 07, 2021 at 09:46:53AM +0100, Tixy wrote:
> On Fri, 2021-05-07 at 09:39 +0200, to...@tuxteam.de wrote:
> >  - 1 I don't understand (who is .pwd.lock [...])

> I thought that was explained on this list a few days ago [...]

Oh, thanks. Must have missed that part :)

Cheers
 - t


signature.asc
Description: Digital signature

Re: OT: dotfile generalities (was Re: repeated system mail, /etc/.pwd.lock ?)

[Tixy]

On Fri, 2021-05-07 at 09:39 +0200, to...@tuxteam.de wrote:
>  - 1 I don't understand (who is .pwd.lock and what is she
>    doing in my /etc? [1]).

I thought that was explained on this list a few days ago (and 17 years
ago! [2]). The file is used by lckpwdf() function [3][4] which is used
to serialise modification to /etc/passed and friends.

> [1] 
> https://www.reddit.com/r/shittyaskreddit/comments/7mq5f9/who_is_general_failure_and_why_is_he_reading_my/

[2] https://lists.debian.org/debian-user/2005/07/msg02949.html
[3] https://linux.die.net/man/3/lckpwdf
[4] 
http://sourceware.org/git/?p=glibc.git;a=blob;f=shadow/lckpwdf.c;hb=284128f68f27567f9cad0078c97d7d807475e0a7


-- 
Tixy

Re: OT: dotfile generalities (was Re: repeated system mail, /etc/.pwd.lock ?)

[tomas]

On Thu, May 06, 2021 at 11:21:14PM -0500, David Wright wrote:

[...]

> Well, here are the files on my system:
> 
> #  find /etc -type f -name '.*' -ls | sort -k 11
>  -rwx-- 84357 May  6 08:25 /etc/.etckeeper
>  -rw---   932 Apr  3  2020 /etc/.gitignore
>  -rw-r--r-- 0 Apr  3  2020 /etc/.java/.systemPrefs/.system.lock
>  -rw-r--r-- 0 Apr  3  2020 /etc/.java/.systemPrefs/.systemRootModFile
>  -rw--- 0 Apr  3  2020 /etc/.pwd.lock
>  -rw-r--r--   102 Jun 23  2019 /etc/cron.d/.placeholder
>  -rw-r--r--   102 Jun 23  2019 /etc/cron.daily/.placeholder
>  -rw-r--r--   102 Jun 23  2019 /etc/cron.hourly/.placeholder
>  -rw-r--r--   102 Jun 23  2019 /etc/cron.monthly/.placeholder
>  -rw-r--r--   102 Jun 23  2019 /etc/cron.weekly/.placeholder
>  -rw-r--r-- 0 Dec 19  2018 /etc/sensors.d/.placeholder
>  -rw-r--r--   220 Apr 17  2019 /etc/skel/.bash_logout
>  -rw-r--r--  3526 Apr 17  2019 /etc/skel/.bashrc
>  -rw-r--r--   807 Apr 17  2019 /etc/skel/.profile
> # 

Nice stats :)

So those are 14 things.

 - 8 of them I'd attribute to etckeeper (the .placeholder ones,
   because git never grokked empty directories)
 - 3 are just part of a directory template (skel)
 - 2 (well, it's actuall a fractal, as usual in that culture)
   are "some Java stuff"
 - 1 I don't understand (who is .pwd.lock and what is she
   doing in my /etc? [1]).

Cheers

[1] 
https://www.reddit.com/r/shittyaskreddit/comments/7mq5f9/who_is_general_failure_and_why_is_he_reading_my/

 - t


signature.asc
Description: Digital signature

Re: OT: dotfile generalities (was Re: repeated system mail, /etc/.pwd.lock ?)

[David Wright]

On Thu 06 May 2021 at 10:46:03 (-0400), Greg Wooledge wrote:
> On Thu, May 06, 2021 at 02:37:17PM +, davidson wrote:
> > > $ shopt -s globstar
> > > $ ls /etc/**/.[^.]*
> > 
> > It now occurs to me that this still omits files like /etc/.a and
> > /etc/..metadotfile
> 
> It doesn't omit .a .  The * is allowed to match the empty string.
> 
> > Instead,
> > 
> >  $ ls /etc/**/.{.?,[^.]}*
> > 
> > does what I intend, which is to display all dotfiles under /etc. Seems
> > unduly complicated if you ask me, but it is what it is.
> 
> find /etc -type f -name '.*'

Well, here are the files on my system:

#  find /etc -type f -name '.*' -ls | sort -k 11
 -rwx-- 84357 May  6 08:25 /etc/.etckeeper
 -rw---   932 Apr  3  2020 /etc/.gitignore
 -rw-r--r-- 0 Apr  3  2020 /etc/.java/.systemPrefs/.system.lock
 -rw-r--r-- 0 Apr  3  2020 /etc/.java/.systemPrefs/.systemRootModFile
 -rw--- 0 Apr  3  2020 /etc/.pwd.lock
 -rw-r--r--   102 Jun 23  2019 /etc/cron.d/.placeholder
 -rw-r--r--   102 Jun 23  2019 /etc/cron.daily/.placeholder
 -rw-r--r--   102 Jun 23  2019 /etc/cron.hourly/.placeholder
 -rw-r--r--   102 Jun 23  2019 /etc/cron.monthly/.placeholder
 -rw-r--r--   102 Jun 23  2019 /etc/cron.weekly/.placeholder
 -rw-r--r-- 0 Dec 19  2018 /etc/sensors.d/.placeholder
 -rw-r--r--   220 Apr 17  2019 /etc/skel/.bash_logout
 -rw-r--r--  3526 Apr 17  2019 /etc/skel/.bashrc
 -rw-r--r--   807 Apr 17  2019 /etc/skel/.profile
# 

(I removed the node#, usage, #links and ownership for brevity.)

I have no argument with /etc/skel/ and the placeholders, obviously.
/etc/.etckeeper is a script for creating directories and resetting
ownerships/permissions in /etc/, and /etc/.gitignore is a list of
filename patterns that etckeeper is configured not to track.

Turning to the dot-directories with:

  # ls -GlgAR /etc/\.[^\.]*/   (pruned and attached)

/etc/.git/ certainly contains configuration files as well as the
repository itself. I know nothing about java, and how /etc/.java
relates to /etc/java/.

Most of this stuff is protected from reading, but I don't see the
point in "hiding" them with dots.

Cheers,
David.
/etc/.git/:
total 244
-rw-r--r--   1 17 May  6 08:25 COMMIT_EDITMSG
-rw-r--r--   1 23 Apr  3  2020 HEAD
drwxr-xr-x   2   4096 Apr  3  2020 branches
-rw-r--r--   1 92 Apr  3  2020 config
-rw-r--r--   1 21 Apr  3  2020 description
drwxr-xr-x   2   4096 Apr  3  2020 hooks
-rw-r--r--   1 206363 May  6 08:35 index
drwxr-xr-x   2   4096 Apr  3  2020 info
drwxr-xr-x   3   4096 Apr  3  2020 logs
drwxr-xr-x 260   4096 Apr  3  2020 objects
drwxr-xr-x   4   4096 Apr  3  2020 refs

/etc/.git/branches:
total 0

/etc/.git/hooks:
total 52
-rwxr-xr-x 1  478 Apr  3  2020 applypatch-msg.sample
-rwxr-xr-x 1  896 Apr  3  2020 commit-msg.sample
-rwxr-xr-x 1 3327 Apr  3  2020 fsmonitor-watchman.sample
-rwxr-xr-x 1  189 Apr  3  2020 post-update.sample
-rwxr-xr-x 1  424 Apr  3  2020 pre-applypatch.sample
-rwxr-xr-x 1  118 Apr  3  2020 pre-commit
-rwxr-xr-x 1 1638 Apr  3  2020 pre-commit.sample
-rwxr-xr-x 1 1348 Apr  3  2020 pre-push.sample
-rwxr-xr-x 1 4898 Apr  3  2020 pre-rebase.sample
-rwxr-xr-x 1  544 Apr  3  2020 pre-receive.sample
-rwxr-xr-x 1 1492 Apr  3  2020 prepare-commit-msg.sample
-rwxr-xr-x 1 3610 Apr  3  2020 update.sample

/etc/.git/info:
total 4
-rw-r--r-- 1 240 Apr  3  2020 exclude

/etc/.git/logs:
total 68
-rw-r--r-- 1 60021 May  6 08:25 HEAD
drwxr-xr-x 3  4096 Apr  3  2020 refs

/etc/.git/logs/refs:
total 4
drwxr-xr-x 2 4096 Apr  3  2020 heads

/etc/.git/logs/refs/heads:
total 64
-rw-r--r-- 1 60021 May  6 08:25 master

/etc/.git/objects:
total 1032
drwxr-xr-x 2 4096 Mar 27 13:03 00
[ … ]
drwxr-xr-x 2 4096 Jun 12  2020 ff
drwxr-xr-x 2 4096 Apr  3  2020 info
drwxr-xr-x 2 4096 Apr  3  2020 pack

/etc/.git/objects/00:
total 92
-r--r--r-- 1 13623 Jun 30  2020 1c2271807b14845a18e475364a48d9652a1ddd
[ … ]
-r--r--r-- 1 2697 Apr  3  2020 fac6f7df55d080c9804ea3386634bb906c531b

/etc/.git/objects/ff:
total 68
-r--r--r-- 1   62 Apr  3  2020 1f5877865a736c3c30efb1d8826aff909e078e
[ … ]
-r--r--r-- 1  380 Apr  3  2020 f8d813e0e033163e1ffe916f4cb8f0fd97f54b

/etc/.git/objects/info:
total 0

/etc/.git/objects/pack:
total 0

/etc/.git/refs:
total 8
drwxr-xr-x 2 4096 May  6 08:25 heads
drwxr-xr-x 2 4096 Apr  3  2020 tags

/etc/.git/refs/heads:
total 4
-rw-r--r-- 1 41 May  6 08:25 master

/etc/.git/refs/tags:
total 0

/etc/.java/:
total 4
drwxr-xr-x 2 4096 Apr  3  2020 .systemPrefs

/etc/.java/.systemPrefs:
total 0
-rw-r--r-- 1 0 Apr  3  2020 .system.lock
-rw-r--r-- 1 0 Apr  3  2020 .systemRootModFile

Re: OT: dotfile generalities (was Re: repeated system mail, /etc/.pwd.lock ?)

[Greg Wooledge]

On Thu, May 06, 2021 at 02:37:17PM +, davidson wrote:
> > $ shopt -s globstar
> > $ ls /etc/**/.[^.]*
> 
> It now occurs to me that this still omits files like /etc/.a and
> /etc/..metadotfile

It doesn't omit .a .  The * is allowed to match the empty string.

> Instead,
> 
>  $ ls /etc/**/.{.?,[^.]}*
> 
> does what I intend, which is to display all dotfiles under /etc. Seems
> unduly complicated if you ask me, but it is what it is.

find /etc -type f -name '.*'

OT: dotfile generalities (was Re: repeated system mail, /etc/.pwd.lock ?)

[davidson]

On Thu, 6 May 2021 davidson wrote:
[dd]

To that end, I can occasionally do something like

$ ls -Rp | less

and make a point of examining the first couple of things that look
unfamiliar.

This misses out dotfiles.

[dd]

So when I look for what I'm missing out on, and do...

$ shopt -s globstar
$ ls /etc/**/.[^.]*


It now occurs to me that this still omits files like /etc/.a and
/etc/..metadotfile

Instead,

 $ ls /etc/**/.{.?,[^.]}*

does what I intend, which is to display all dotfiles under /etc. Seems
unduly complicated if you ask me, but it is what it is.

I can feel it coming on: Someone is going to tell me to just use find.

Talk about unduly complicated! And I don't want to.



...then I find

* /etc/.pwd.lock, clearly an exception to the /etc profile above
* various ".placeholder" files, also clear exceptions
* model (eponymous) home directory config files under /etc/skel
* some ".depends.*" files in /etc/init.d/ which are not init scripts

[dd]

--
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

Re: repeated system mail, /etc/.pwd.lock ?

[davidson]

On Wed, 5 May 2021 David Wright wrote:
[dd]


One thing I didn't learn is why .pwd.lock is in /etc/ rather than,
say, /run/lock/. Perhaps related, why are there dotfiles in /etc/
anyway. (.git/, .java/, .etckeeper, .gitignore are the others.)
What are they hiding from?

[dd]

I would assume that they are "hidden" because they are merely
peripheral or auxiliary to the *purpose* of the directory containing
them.

We tend to group files into directories devoted to one function or
another. This certainly seems to be the unstated premise of hier(7),
for example.

So home directories hold a user's content. Per-user configuration
falls outside that canonical function, hence the ~/.dotfile convention
in home directories.

Likewise, I assume that files under /etc are text files that determine
system configuration, and that if system configuration is something I
care about, then I ought to at least gain a passing familiarity with
each such file under /etc, ideally (eventually) become familiar with
its content and function, and generally consider myself responsible
for the conseqences of that content, take responsiblity for reviewing
and editing it, etc.

To that end, I can occasionally do something like

 $ ls -Rp | less

and make a point of examining the first couple of things that look
unfamiliar.

This misses out dotfiles. But with a few exceptions --which seem to
prove the rule after all-- this omission serves my purposes just
fine. That is, when I understand the purpose of the file hierarchy
then I know where to look to find what I want, and I'm spared the
trouble of ignoring certain files whose presence is
helpful-yet-peripheral to the purpose of a given directory.

So when I look for what I'm missing out on, and do...

 $ shopt -s globstar
 $ ls /etc/**/.[^.]*

...then I find

 * /etc/.pwd.lock, clearly an exception to the /etc profile above
 * various ".placeholder" files, also clear exceptions
 * model (eponymous) home directory config files under /etc/skel
 * some ".depends.*" files in /etc/init.d/ which are not init scripts

I don't have under /etc any of .git/, .gitignore, .etckeeper, .java/.
But don't they contain mere meta-info, relative to the primary purpose
of the directory containing them? (I don't know a thing about .java/
btw, so am curious.)

If I put say /etc/default/ under revision control, I would not want
new meta-files appearing whenever I did "ls /etc/default".

--
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

Re: repeated system mail, /etc/.pwd.lock ?

[David Wright]

On Wed 05 May 2021 at 12:01:07 (-0400), Greg Wooledge wrote:
> On Wed, May 05, 2021 at 10:36:53AM -0500, David Wright wrote:
> > OTOH perhaps monkeysphere has some reason to lock /etc/passwd et al
> > during operation. Running strings on its binaries might throw up
> > some 'pwd.lock' matches. Or one could inotifywatch the program to
> > see how often it is run (unless it's a daemon). Just thinking aloud.
> 
> I actually looked for that filename in "strings /usr/bin/sudo" and
> "strings /usr/sbin/vipw" before Google told me that it's used by those
> libc functions (lckpwdf(3) and its buddy ulckpwdf(3)).
> 
> It's a bit odd that the man page doesn't contain the filename.  Usually
> you'd expect it to.
> 
> > One thing I didn't learn is why .pwd.lock is in /etc/ rather than,
> > say, /run/lock/. Perhaps related, why are there dotfiles in /etc/
> > anyway. (.git/, .java/, .etckeeper, .gitignore are the others.)
> > What are they hiding from?
> 
> This predates /run by a long time.
> 
> Also, presumably, it wants to be in /etc because that's where the file
> that it's paired with lives.  Dot-lock files are usually in the same
> location as the files they represent.

Yes—though its name, in fact, ensures that it never lists next to any
of the files that it might be assumed to lock.

> Why the dot?  So that it doesn't show up in a casual "ls" and cause a
> bunch of n00b questions, obviously.

Yes, I can understand that for the users' home directories; they don't
want listings of their own files to be interspersed with configuration
files. And many of the latter files will be read and written on behalf
of the user rather than by themselves, so there's little need for them
to see them all the time.

But /etc isn't anybody's home directory: it's configuration information
for the whole system. So there's no need to dot any of the files and
directories in order to either inform the sysadmin that these are
configuration files, or to hide them. Anyway, that was my (related) point.

Cheers,
David.

Re: repeated system mail, /etc/.pwd.lock ?

[Charles Curley]

On Wed, 5 May 2021 10:36:53 -0500
David Wright  wrote:

> [W]hy are there dotfiles in /etc/ anyway.
> (.git/, .java/, .etckeeper, .gitignore are the others.) What are they
> hiding from?

Indeed. And shouldn't one be backing them up?

I see that amanda has been backing them up and that "add *" in
amrecover does add them to the extract list (phew!). Would other backup
tools catch dotfiles?

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

David Wright wrote:

> Yes, it was an assumption, and perhaps now we shall never
> know. (Sampling the emails didn't appeasr to be an option.)
> We also were not told whether 2757 notifications came in
> over a week, a month, a year, or since openssh-client was
> installed, whenever that was (possibly at installation).

Oh, no, I removed mails all the time, they came at least every
10 minutes or so is what I would approximate from the holster.

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

Greg Wooledge wrote:

> I interpreted it as literally being thousands of instances
> of the *same* file, the one shown in the Subject: header and
> in the original message body.

They were all from iwatch, but they were so many I don't know
if they were exactly the same, and now I don't get any (mails)
anymore. Good :)

> (Also I'd never heard of "monkeysphere" before and didn't
> even know that openssh-client suggested it. So it's been an
> educational thread.)

Yes, thanks everyone :)

I learned something as well, how to delete mails. First see
how many mails there are, say there are 756, then type t 1-756
RET and then hold down q :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[Greg Wooledge]

On Wed, May 05, 2021 at 10:36:53AM -0500, David Wright wrote:
> OTOH perhaps monkeysphere has some reason to lock /etc/passwd et al
> during operation. Running strings on its binaries might throw up
> some 'pwd.lock' matches. Or one could inotifywatch the program to
> see how often it is run (unless it's a daemon). Just thinking aloud.

I actually looked for that filename in "strings /usr/bin/sudo" and
"strings /usr/sbin/vipw" before Google told me that it's used by those
libc functions (lckpwdf(3) and its buddy ulckpwdf(3)).

It's a bit odd that the man page doesn't contain the filename.  Usually
you'd expect it to.

> One thing I didn't learn is why .pwd.lock is in /etc/ rather than,
> say, /run/lock/. Perhaps related, why are there dotfiles in /etc/
> anyway. (.git/, .java/, .etckeeper, .gitignore are the others.)
> What are they hiding from?

This predates /run by a long time.

Also, presumably, it wants to be in /etc because that's where the file
that it's paired with lives.  Dot-lock files are usually in the same
location as the files they represent.

Why the dot?  So that it doesn't show up in a casual "ls" and cause a
bunch of n00b questions, obviously.

Re: repeated system mail, /etc/.pwd.lock ?

[David Wright]

On Wed 05 May 2021 at 07:26:34 (-0400), Greg Wooledge wrote:
> On Tue, May 04, 2021 at 09:32:49PM -0500, David Wright wrote:
> > It looks reasonable for determining whether your system files are
> > being interfered with. But you just showed one example from the
> > log, which was for the /etc/.pwd.lock lockfile. I assume you don't
> > have 2757 of these but, rather, the names of an assortment of files.
> 
> That's an interesting interpretation.  If that's actually *true*, I
> wish the OP had made that more clear.  I interpreted it as literally
> being thousands of instances of the *same* file, the one shown in the
> Subject: header and in the original message body.

Yes, it was an assumption, and perhaps now we shall never know.
(Sampling the emails didn't appeasr to be an option.)
We also were not told whether 2757 notifications came in over
a week, a month, a year, or since openssh-client was installed,
whenever that was (possibly at installation).

  $ zgrep ':[0-9][0-9] configure ' /var/log/dpkg.log* | sort -k 4 | less
run on this system, which has just passed its first birthday¹, shows
2788 lines, and each must represent a number of modifications to
the directories given earlier (/etc, /[s]bin, /lib). So 2757 looks
small in that context.

OTOH perhaps monkeysphere has some reason to lock /etc/passwd et al
during operation. Running strings on its binaries might throw up
some 'pwd.lock' matches. Or one could inotifywatch the program to
see how often it is run (unless it's a daemon). Just thinking aloud.

> (In which case, removing iwatch will certainly stop the logging, but
> it won't stop whoever is locking and unlocking your passwd/shadow
> files thousands of times, which is something I might care enough to
> investigate -- and is a great reason for installing iwatch, to look for
> such a thing.)
> 
> (Also I'd never heard of "monkeysphere" before and didn't even know
> that openssh-client suggested it.  So it's been an educational thread.)

One thing I didn't learn is why .pwd.lock is in /etc/ rather than,
say, /run/lock/. Perhaps related, why are there dotfiles in /etc/
anyway. (.git/, .java/, .etckeeper, .gitignore are the others.)
What are they hiding from?

¹ alternatives, apt, and dpkg have 5yr log rotation, exim has 10.

Cheers,
David.

Re: repeated system mail, /etc/.pwd.lock ?

[Greg Wooledge]

On Tue, May 04, 2021 at 09:32:49PM -0500, David Wright wrote:
> It looks reasonable for determining whether your system files are
> being interfered with. But you just showed one example from the
> log, which was for the /etc/.pwd.lock lockfile. I assume you don't
> have 2757 of these but, rather, the names of an assortment of files.

That's an interesting interpretation.  If that's actually *true*, I
wish the OP had made that more clear.  I interpreted it as literally
being thousands of instances of the *same* file, the one shown in the
Subject: header and in the original message body.

(In which case, removing iwatch will certainly stop the logging, but
it won't stop whoever is locking and unlocking your passwd/shadow
files thousands of times, which is something I might care enough to
investigate -- and is a great reason for installing iwatch, to look for
such a thing.)

(Also I'd never heard of "monkeysphere" before and didn't even know
that openssh-client suggested it.  So it's been an educational thread.)

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

David Wright wrote:

> Myself, I find inotify-tools more useful: I use inotifywait
> in a loop, waiting for a browser to close files in its
> cache. I then examine their filetype and copy the ones
> I want, giving them sensible (timestamp) names. Very useful
> for capturing (typically, live) video.
>
> But I can see the usefulness of iwatch if you're maintaining
> a website or a server farm, as in their examples. (My random
> collection of PCs only constitutes a smallholding.)

Yeah, I'm sure it is a good tool when it works but here
something has gone haywire so better remove it...

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

David Wright wrote:

> might help determine why you installed iwatch.

Oh, so I did?

Well then, I'll just remove it!

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

David Wright wrote:

> $ aptitude why iwatch

i   openssh-client Suggests   monkeysphere 
i A monkeysphere   Suggests   monkeysphere-validation-agent
i A msva-perl  Provides   monkeysphere-validation-agent
i A msva-perl  Recommends liblinux-inotify2-perl   
i A liblinux-inotify2-perl Suggests   iwatch   

> $ zgrep -B1 iwatch /var/log/apt/history.log*

/var/log/apt/history.log.2.gz:Requested-By: incal (1000)

/var/log/apt/history.log.2.gz:Install: openvpn:amd64 (2.5.1-1,
automatic), network-manager-openvpn-gnome:amd64 (1.8.12-2,
automatic), python3-appdirs:amd64 (1.4.4-1, automatic),
dnsmasq:amd64 (2.84-1, automatic), ssh-askpass:amd64
(1:1.2.4.1-10+b1, automatic), pcmciautils:amd64 (018-12,
automatic), librygel-renderer-gst-2.6-2:amd64 (0.40.0-1,
automatic), libnet-cidr-perl:amd64 (0.20-1, automatic),
rygel-playbin:amd64 (0.40.0-1, automatic), easy-rsa:amd64
(3.0.8-1, automatic), network-manager-openvpn:amd64 (1.8.12-2,
automatic), libgupnp-dlna-2.0-3:amd64 (0.10.5-4, automatic),
iwatch:amd64 (0.2.2-9, automatic), agent-transfer:amd64
(0.43-3.1, automatic), apg:amd64 (2.2.3.dfsg.1-5+b2,
automatic), network-manager-openconnect-gnome:amd64 (1.2.6-1,
automatic), packagekit-tools:amd64 (1.2.2-2, automatic),
gnome-software:amd64 (3.38.1-1, automatic),
dbus-user-session:amd64 (1.12.20-2, automatic),
libnss-myhostname:amd64 (247.3-3, automatic),
python3-axolotl-curve25519:amd64 (0.4.1.post2-2+b4,
automatic), libstoken1:amd64 (0.92-1, automatic),
libtype-tiny-xs-perl:amd64 (0.022-1, automatic),
malcontent:amd64 (0.10.0-2, automatic), libnl-cli-3-200:amd64
(3.4.0-1+b1, automatic), librygel-core-2.6-2:amd64 (0.40.0-1,
automatic), librygel-ruih-2.0-1:amd64 (0.40.0-1, automatic),
gnome-software-common:amd64 (3.38.1-1, automatic),
libmoox-late-perl:amd64 (0.100-1, automatic),
apt-config-icons-hidpi:amd64 (0.14.2-1, automatic),
usbguard:amd64 (1.0.0+ds-2, automatic), libmutter-7-0:amd64
(3.38.3-5, automatic), rygel-preferences:amd64 (0.40.0-1,
automatic), python3-transitions:amd64 (0.8.6-1, automatic),
pptp-linux:amd64 (1.10.0-1, automatic),
libio-socket-socks-perl:amd64 (0.74-1.1, automatic),
python3-yowsup:amd64 (3.2.3-2, automatic), ufw:amd64
(0.36-7.1, automatic), libcrypt-x509-perl:amd64 (0.53-1,
automatic), libmalcontent-ui-0-0:amd64 (0.10.0-2, automatic),
libxml-stream-perl:amd64 (1.24-4, automatic),
libmalcontent-0-0:amd64 (0.10.0-2, automatic),
gstreamer1.0-clutter-3.0:amd64 (3.0.27-2, automatic),
xdg-desktop-portal-gtk:amd64 (1.8.0-1, automatic),
python3-dissononce:amd64 (0.34.3-2, automatic),
molly-guard:amd64 (0.7.2, automatic), libsnapd-glib1:amd64
(1.58-4, automatic), libostree-1-1:amd64 (2020.8-2,
automatic), libcolorhug2:amd64 (1.4.5-3, automatic),
resolvconf:amd64 (1.87, automatic), vpnc:amd64
(0.5.3+git20210125-1, automatic), wireless-tools:amd64
(30~pre9-13.1, automatic), colord:amd64 (1.4.5-3, automatic),
software-properties-common:amd64 (0.96.20.2-2.1, automatic),
monkeysphere:amd64 (0.43-3.1, automatic), libgsound0:amd64
(1.0.2-5, automatic), libtomcrypt1:amd64 (1.18.2-5,
automatic), openssh-server:amd64 (1:8.4p1-5, automatic),
libio-multiplex-perl:amd64 (1.16-1.1, automatic),
wireless-regdb:amd64 (2020.04.29-2, automatic), pipewire:amd64
(0.3.19-4, automatic), tumbler:amd64 (4.16.0-1, automatic),
libcheese-gtk25:amd64 (3.38.0-3, automatic),
python3-software-properties:amd64 (0.96.20.2-2.1, automatic),
malcontent-gui:amd64 (0.10.0-2, automatic),
librygel-server-2.6-2:amd64 (0.40.0-1, automatic),
libges-1.0-0:amd64 (1.18.3-1, automatic), squashfs-tools:amd64
(1:4.4-2, automatic), yowsup-cli:amd64 (3.2.3-2, automatic),
tumbler-common:amd64 (4.16.0-1, automatic),
mutter-common:amd64 (3.38.3-5, automatic),
gir1.2-packagekitglib-1.0:amd64 (1.2.2-2, automatic),
libusbguard0:amd64 (1.0.0+ds-2, automatic),
network-manager-gnome:amd64 (1.20.0-3, automatic),
libgee-0.8-2:amd64 (0.20.3-1, automatic), libnma-common:amd64
(1.8.30-1, automatic), network-manager-openconnect:amd64
(1.2.6-1, automatic), gnome-settings-daemon-common:amd64
(3.38.1-3, automatic), libappstream4:amd64 (0.14.2-1,
automatic), libnet-server-perl:amd64 (2.009-2, automatic),
appstream:amd64 (0.14.2-1, automatic),
gnome-software-plugin-snap:amd64 (3.38.1-1, automatic),
network-manager-vpnc-gnome:amd64 (1.2.6-3, automatic),
packagekit:amd64 (1.2.2-2, automatic),
libgnupg-interface-perl:amd64 (1.01-2, automatic),
python3-consonance:amd64 (0.1.3-3, automatic),
network-manager-pptp-gnome:amd64 (1.2.8-3+b2, automatic),
libappstream-glib8:amd64 (0.7.18-1, automatic), libccid:amd64
(1.4.34-1, automatic), lockfile-progs:amd64 (0.1.18,
automatic), apt-config-icons:amd64 (0.14.2-1, automatic),
libtype-tiny-perl:amd64 (1.012001-2, automatic), libndp0:amd64
(1.6-1+b1, automatic), libsub-handlesvia-perl:amd64 (0.016-1,
automatic), gnome-settings-daemon:amd64 (3.38.1-3, automatic),
libteam5:amd64 (1.31-1, automatic), policykit-1-gnome:amd64
(0.105-7, automatic), 

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

Kushal Kumaran wrote:

> The manpage at
> https://manpages.debian.org/buster/iwatch/iwatch.1.en.html
> shows log output similar to what you see. Check your iwatch
> configuration and see what it is doing.

Thanks, but I've never heard of iwatch, so I haven't mucked
around with its config file. But OK, here it is





  
  
Operating System

/bin
/sbin
/etc
/lib
/lib/modules
  


Does that say anything to you?

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[David Wright]

On Wed 05 May 2021 at 04:39:21 (+0200), Emanuel Berg wrote:
> David Wright wrote:
> 
> > might help determine why you installed iwatch.
> 
> Oh, so I did?
> 
> Well then, I'll just remove it!

Myself, I find inotify-tools more useful: I use inotifywait in a loop,
waiting for a browser to close files in its cache. I then examine
their filetype and copy the ones I want, giving them sensible
(timestamp) names. Very useful for capturing (typically, live) video.

But I can see the usefulness of iwatch if you're maintaining a
website or a server farm, as in their examples. (My random collection
of PCs only constitutes a smallholding.)

Cheers,
David.

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

>> The "IN_" prefix tells you that this is an inotify event.
>> IN_CLOSE_WRITE fires when a process _had_ the specified
>> file open for writing, but has just closed it. Perhaps you
>> have an "incron" job somewhere?
>
> I have cron do two very short scripts every @midnight, these
> run fine individually and I've not experienced any problems
> at 00:00 so I take it they run fine there as well.
>
> Other than that I have not fiddled with cronjobs at all,
> I think. (?)

No one has any clue how to debug this? Really?

How about directing all them mails to /dev/disintegrate then?

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[David Wright]

On Wed 05 May 2021 at 03:11:53 (+0200), Emanuel Berg wrote:
> Kushal Kumaran wrote:
> 
> > The manpage at
> > https://manpages.debian.org/buster/iwatch/iwatch.1.en.html
> > shows log output similar to what you see. Check your iwatch
> > configuration and see what it is doing.
> 
> Thanks, but I've never heard of iwatch, so I haven't mucked
> around with its config file. But OK, here it is

[ … configuration … ]

> Does that say anything to you?

It looks reasonable for determining whether your system files are
being interfered with. But you just showed one example from the
log, which was for the /etc/.pwd.lock lockfile. I assume you don't
have 2757 of these but, rather, the names of an assortment of files.

So—you're interfering with the system, by installing, removing and
reconfiguring software; people are changing their passwords, and
so on.

You might be able to correlate the timestamps of some of the changes
being made with those in logs like /var/log/apt/history*, which
shows software upgrades.

/etc/.pwd.lock is probably not a good example, as it's opened for
writing but not written to. On my system, it's the 3rd-oldest
file modification in /etc (after /etc/{shells,environment} and
before /etc/{adduser.conf,machine-id}).

BTW,

  $ aptitude why iwatch

or

  $ zgrep -B1 iwatch /var/log/apt/history.log*

might help determine why you installed iwatch.

Cheers,
David.

Re: repeated system mail, /etc/.pwd.lock ?

[Greg Wooledge]

On Wed, May 05, 2021 at 03:11:53AM +0200, Emanuel Berg wrote:
> Kushal Kumaran wrote:
> > The manpage at
> > https://manpages.debian.org/buster/iwatch/iwatch.1.en.html
> > shows log output similar to what you see. Check your iwatch
> > configuration and see what it is doing.
> 
> Thanks, but I've never heard of iwatch, so I haven't mucked
> around with its config file. But OK, here it is
> 
>   
> /etc

> Does that say anything to you?

It says that it's watching /etc.  Which is where your file is.  No
surprises here so far.

What you really should be asking is, "What process is using this file,
and why don't I know about it?"

Googling for the file name turns up this:

https://lists.debian.org/debian-user/2005/07/msg02949.html

  The lckpwdf() and ulckpwdf() functions enable modification access to the
  password databases through the lock file. A process first uses lckpwdf() to
  lock the lock file, thereby gaining exclusive rights to modify the
  /etc/passwd or /etc/shadow password database. (See "passwd"  and "shadow" man
  pages.)

  Upon completing modifications, a process should release the lock on the lock
  file using ulckpwdf(). This mechanism prevents simultaneous modification of
  the password databases. The lock file, /etc/.pwd.lock, is used to coordinate
  modification access to the password databases /etc/passwd and /etc/shadow.

So, as one might expect just from the file name, it's a lock file for
the passwd file.

Again, the real question is: What process is touching your passwd file
at this time, and why don't you know about it?

Re: repeated system mail, /etc/.pwd.lock ?

[Kushal Kumaran]

On Tue, May 04 2021 at 11:11:02 AM, Emanuel Berg  wrote:
> Darac Marjal wrote:
>
>> The "IN_" prefix tells you that this is an inotify event.
>> IN_CLOSE_WRITE fires when a process _had_ the specified file
>> open for writing, but has just closed it. Perhaps you have
>> an "incron" job somewhere?
>
> I have cron do two very short scripts every @midnight, these
> run fine individually and I've not experienced any problems at
> 00:00 so I take it they run fine there as well.
>
> Other than that I have not fiddled with cronjobs at all,
> I think. (?)
>
> This is what 'ps -e' tells me:
>
>  9 ~/ppl/mats ps -e
> PID TTY  TIME CMD
> ... snip
>1149 ?00:00:00 iwatch
> ... snip

The manpage at
https://manpages.debian.org/buster/iwatch/iwatch.1.en.html shows log
output similar to what you see.  Check your iwatch configuration and see
what it is doing.

-- 
regards,
kushal

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

Darac Marjal wrote:

> The "IN_" prefix tells you that this is an inotify event.
> IN_CLOSE_WRITE fires when a process _had_ the specified file
> open for writing, but has just closed it. Perhaps you have
> an "incron" job somewhere?

I have cron do two very short scripts every @midnight, these
run fine individually and I've not experienced any problems at
00:00 so I take it they run fine there as well.

Other than that I have not fiddled with cronjobs at all,
I think. (?)

This is what 'ps -e' tells me:

 9 ~/ppl/mats ps -e
PID TTY  TIME CMD
  1 ?00:00:26 systemd
  2 ?00:00:00 kthreadd
  3 ?00:00:00 rcu_gp
  4 ?00:00:00 rcu_par_gp
  6 ?00:00:00 kworker/0:0H-kblockd
  8 ?00:00:00 mm_percpu_wq
  9 ?00:00:01 ksoftirqd/0
 10 ?00:00:28 rcu_sched
 11 ?00:00:00 rcu_bh
 12 ?00:00:00 migration/0
 14 ?00:00:00 cpuhp/0
 15 ?00:00:00 cpuhp/1
 16 ?00:00:00 migration/1
 17 ?00:00:00 ksoftirqd/1
 19 ?00:00:00 kworker/1:0H-kblockd
 20 ?00:00:00 cpuhp/2
 21 ?00:00:00 migration/2
 22 ?00:00:01 ksoftirqd/2
 24 ?00:00:00 kworker/2:0H-kblockd
 25 ?00:00:00 cpuhp/3
 26 ?00:00:00 migration/3
 27 ?00:00:00 ksoftirqd/3
 29 ?00:00:00 kworker/3:0H-kblockd
 30 ?00:00:00 kdevtmpfs
 31 ?00:00:00 netns
 32 ?00:00:00 kauditd
 35 ?00:00:00 khungtaskd
 36 ?00:00:00 oom_reaper
 37 ?00:00:00 writeback
 38 ?00:00:00 kcompactd0
 39 ?00:00:00 ksmd
 40 ?00:00:01 khugepaged
 41 ?00:00:00 crypto
 42 ?00:00:00 kintegrityd
 43 ?00:00:00 kblockd
 44 ?00:00:00 edac-poller
 46 ?00:00:00 devfreq_wq
 47 ?00:00:00 watchdogd
 49 ?00:00:00 irq/25-AMD-Vi
 50 ?00:00:05 kswapd0
 68 ?00:00:00 kthrotld
 70 ?00:00:00 amd_iommu_v2
 71 ?00:00:00 ipv6_addrconf
 80 ?00:00:00 kstrp
132 ?00:00:00 nvme-wq
133 ?00:00:00 nvme-reset-wq
134 ?00:00:00 nvme-delete-wq
147 ?00:00:00 ata_sff
148 ?00:00:00 scsi_eh_0
149 ?00:00:00 scsi_tmf_0
150 ?00:00:00 scsi_eh_1
151 ?00:00:00 scsi_tmf_1
153 ?00:00:00 scsi_eh_2
154 ?00:00:00 scsi_tmf_2
155 ?00:00:00 scsi_eh_3
156 ?00:00:00 scsi_tmf_3
157 ?00:00:00 scsi_eh_4
158 ?00:00:00 scsi_tmf_4
159 ?00:00:00 scsi_eh_5
160 ?00:00:00 scsi_tmf_5
161 ?00:00:00 scsi_eh_6
162 ?00:00:00 scsi_tmf_6
163 ?00:00:00 scsi_eh_7
164 ?00:00:00 scsi_tmf_7
167 ?00:00:00 scsi_eh_8
168 ?00:00:00 scsi_tmf_8
229 ?00:00:00 kworker/2:1H-kblockd
231 ?00:00:00 kworker/3:1H-kblockd
240 ?00:00:00 md
250 ?00:00:00 raid5wq
285 ?00:00:00 kworker/1:1H-kblockd
304 ?00:00:00 kworker/u65:0
305 ?00:00:02 jbd2/sda2-8
306 ?00:00:00 ext4-rsv-conver
307 ?00:00:00 kworker/0:1H-kblockd
371 ?00:00:02 systemd-journal
407 ?00:00:00 systemd-udevd
410 ?00:00:00 loop0
411 ?00:00:00 loop1
414 ?00:00:00 loop2
415 ?00:00:00 loop3
417 ?00:00:00 loop4
472 ?00:00:00 led_workqueue
475 ?00:00:00 nvkm-disp
515 ?00:00:00 ttm_swap
590 ?00:00:00 dhclient
591 ?00:00:00 rpcbind
592 ?00:00:02 systemd-resolve
596 ?00:00:00 accounts-daemon
598 ?00:00:03 dbus-daemon
599 ?00:00:02 NetworkManager
605 ?00:00:07 snapd
606 ?00:00:18 systemd-logind
608 ?00:00:00 usbguard-dbus
665 ?00:00:01 usbguard-daemon
713 ?00:00:00 polkitd
724 ?00:00:11 ntpd
726 ?00:00:00 sshd
741 tty1 00:00:00 login
   1130 ?00:00:00 exim4
   1149 ?00:00:00 iwatch
   1162 ?00:00:00 systemd
   1163 ?00:00:00 (sd-pam)
   1190 ?00:00:01 pipewire
   1191 ?00:07:31 pulseaudio
   1193 ?00:00:02 tracker-miner-f
   1194 tty1 00:00:00 zsh
   1197 ?00:00:07 dbus-daemon
   1206 ?00:00:00 pipewire-media-
   1210 ?00:00:00 upowerd
   1254 tty1 00:08:35 emacs
   1351 ?00:00:41 tmux: server
   1422 tty6 00:00:00 login
   1431 tty6 00:00:00 zsh
   1444 tty6 00:00:00 startx
   1478 tty6 00:00:00 xinit
   1479 tty6 00:04:21 Xorg
   1519 tty6 00:00:00 sh
   1525 tty6 00:00:00 openbsd-cwm
   1532 tty6 00:00:10 xterm
   1533 pts/200:00:00 tmux: client
   

Re: repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

Darac Marjal wrote:

>> I get system mail all the time - I've got 2757 at the
>> moment - that tells me that
>>
>>   [ 4/Apr/2021 22:11:33]
>>   IN_CLOSE_WRITE /etc/.pwd.lock
>>   * /etc/.pwd.lock is closed
>>
>> Any clues what that problem might be?
>
> The "IN_" prefix tells you that this is an inotify event.
> IN_CLOSE_WRITE fires when a process _had_ the specified file
> open for writing, but has just closed it. Perhaps you have
> an "incron" job somewhere?

I don't know, how do I find out?

-- 
underground experts united
https://dataswamp.org/~incal

repeated system mail, /etc/.pwd.lock ?

[Emanuel Berg]

I get system mail all the time - I've got 2757 at the moment -
that tells me that

  [ 4/Apr/2021 22:11:33]
  IN_CLOSE_WRITE /etc/.pwd.lock
  * /etc/.pwd.lock is closed

Any clues what that problem might be?

TIA

-- 
underground experts united
https://dataswamp.org/~incal

Re: repeated system mail, /etc/.pwd.lock ?

[Darac Marjal]

On 04/05/2021 07:12, Emanuel Berg wrote:
> I get system mail all the time - I've got 2757 at the moment -
> that tells me that
>
>   [ 4/Apr/2021 22:11:33]
>   IN_CLOSE_WRITE /etc/.pwd.lock
>   * /etc/.pwd.lock is closed
>
> Any clues what that problem might be?
The "IN_" prefix tells you that this is an inotify event. IN_CLOSE_WRITE
fires when a process _had_ the specified file open for writing, but has
just closed it. Perhaps you have an "incron" job somewhere?
>
> TIA
>



OpenPGP_signature
Description: OpenPGP digital signature