Re: Why do we have a default activated man-db.timer?

[Tom Bachreier]

Hi Étienne!

Thanks for your comprehensive explanation. I appreciate it very much.

Feb 12, 2019, 1:38 AM by etienne.moll...@mailoo.org:

> Tom Bachreier, on 2019-02-11:
>
>> BUT wait...
>>
>> I was wondering why do we have man-db.timer in the first place?
>>
>
> On my Sid machine, this file is known under the name of
> /usr/lib/systemd/system/man-db.service and has, among other
> things, these lines:
>
Just for the record:
On testing I found the file in /lib/systemd/system instead.

>
>  # Expunge old catman pages which have not been read in a week.
>  ExecStart=/usr/bin/find /var/cache/man -type f -name *.gz -atime +6 -delete
>  # Regenerate man database.
>  ExecStart=/usr/bin/mandb --quiet
>
> Seems there is not only a mandb re-indexing, but also some
> cleanup is done apparently, given the `find [...] -delete`.
>
[Re-indexing for man, whatis and apropos]

In my opinion still the perfect job for a postinstall-script. Manpages
doesn't change other than during install, update or remove of
packages.


>
> The other mechanism consists in caching the binary
> representation of compiled manual pages.  
>
[...]

> You can try for yourself how `man` handles the research inside
> your documentation by passing the debug option -d and browsing
> the output debug.out at your convenience:
>
>  $ man -d man 2> debug.out
>
> I discovered for instance that I never use this cache because I
> cap my terminal width to 65 characters using the variable
> MANWIDTH, and the debug output shown me:
>
>  Terminal width 65 not within cat page range [80, 80]
>
Same here. But I tried a 80 columns terminal, too and no cache files
were generated either. I don't care but it is strange. I do not even have
catx-directories - not in /usr/share/man/ or /var/cache/man/.

The debug shows in this case:
> found ultimate source file /usr/share/man/man1/man.1.gz
> converted /usr/share/man/man1/man.1.gz to /usr/share/man/cat1/man.1.gz
> is_changed: a=/usr/share/man/man1/man.1.gz, b=/usr/share/man/cat1/man.1.gz 
> (-2)
> converted /var/cache/man/cat1/man.1.gz to /var/cache/man/cat1/man.1.gz
> will try cat file /var/cache/man/cat1/man.1.gz
[...]
> is_changed: a=/usr/share/man/man1/man.1.gz, b=/var/cache/man/cat1/man.1.gz 
> (-2)
> cat dir /var/cache/man/cat1 does not exist

I would translate this to: It tried but failed to generate the cache file.

> Since lots of people do not use 80 char width terminals anymore,
> mostly because of "maximize window", I get this function is not
> of much use lately, at least not by using Debian default value.
> That thing can be configured in /etc/manpath.config if you are
> interested in such manipulations.
>
In /etc/manpath.config I found one can disable caching completely - nice.
> NOCACHE

>
> Well, I don't believe this catman mechanism is of much use
> today, most manual pages are rendered in the blink of an eye.
> Last time I had to /wait/ more than a seconds for a manual page
> to be formatted, that was on a Sun Fire V440.
>
I agree, for myself it is pretty useless. I never missed it.

>
> To summarize, the patched service does a clear the cache of
> unused raw man pages, which should not be referenced anymore by
> mandb.  Part of all this is more or less described in the manual
> pages man(1), most particularly in sections "OVERVIEW" and
> "DEFAULTS", which have a succinct descriptions of the various
> mechanisms involved.  See also catman(8) and mandb(8).
>
>
To summarize for myself. The timer/service doesn't do anything valuable
for me so I will disable it.

Étienne, thanks again for your time! It helped me to understand if the
timer is needed on my computers or not.

Tom

Re: Why do we have a default activated man-db.timer?

[Étienne Mollier]

Tom Bachreier, on 2019-02-11:
> With the migration of man-db 2.8.5-1 into testing about a month ago I
> get bothered once a day with a lot of useless lines in syslog:
> > Feb 10 00:00:51 osprey mandb[3543]: Purging old database entries in 
> > /usr/share/man...
> > Feb 10 00:00:51 osprey mandb[3543]: Processing manual pages under 
> > /usr/share/man...
> > [...]
[...]
> Thankfully Francois filed a bug earlier:
>  >
>
> which got fixed upstream:
>   
> >
>
> So the messages will go away for me in a couple of days.
> Problem solved. :-)
>
> BUT wait...
> I was wondering why do we have man-db.timer in the first place?

Good Day Tom,

That is an interesting question with a story behind that deserve
to be told, definitely not obvious and probably easy to miss,
mostly due to the computer power we can afford these days.

Firstly let's have a look at the patch:


https://git.savannah.gnu.org/cgit/man-db.git/commit/?id=a4206c27060357cc78219a54349624e0d0675aff

Modifications have been brought to the service descriptor of
mandb, to regenerate the database of manual pages in quiet mode.
On my Sid machine, this file is known under the name of
/usr/lib/systemd/system/man-db.service and has, among other
things, these lines:

Type=oneshot
# Recover from deletion, per FHS.
ExecStart=+/usr/bin/install -d -o man -g man -m 0755 /var/cache/man
# Expunge old catman pages which have not been read in a week.
ExecStart=/usr/bin/find /var/cache/man -type f -name *.gz -atime +6 
-delete
# Regenerate man database.
ExecStart=/usr/bin/mandb --quiet

Seems there is not only a mandb re-indexing, but also some
cleanup is done apparently, given the `find [...] -delete`.

Manual pages have various mechanisms to speedup fetching and
showing documentation.  For one is this indexing, done by mandb,
gathering the multiple locations of manual pages, such as
/usr/share/man for the system of course, but also locations for
custom programs, such as /usr/local/man, and perhaps you might
wish to append some third parties too, with /opt/man, $HOME/man
or whatever.  Anyway, these indexes do speedup fetching manual
pages meta-information such as:

- where to find the original content of the documentation when
  running `man`:

$ man man

- fetch the short description of commands when running `whatis`:

$ whatis man
man (1)  - an interface to the on-line reference manuals
man (7)  - macros to format man pages

- search for commands by using a few key words when running
  `apropos`:

$ apropos --and create or update manual page
catman (8)   - create or update the pre-formatted manual pages
mandb (8)- create or update the manual page index caches

The other mechanism consists in caching the binary
representation of compiled manual pages.  The original file
containing the documentation, inside /usr/share/man, is
written in groff, or mandoc on BSD, anyway the thing shown
through the pager (which is just either `more` or `less`,
finally, or `most` sometimes) when running `man` is not a direct
interpretation of the plain text file, but an intermediate
binary representation mostly composed of text and terminal
escape sequences (chosen depending on the value of the TERM
variable).  The caching of this intermediate representation is
done through the use of `cat` pages, and their directory tree
/var/cache/man, mimicking the tree below /usr/share/man.  The
only differences between the two are, the directories "man*/"
named "cat*/" instead, and raw manual page representation
instead of plain text files.

You can try for yourself how `man` handles the research inside
your documentation by passing the debug option -d and browsing
the output debug.out at your convenience:

$ man -d man 2> debug.out

I discovered for instance that I never use this cache because I
cap my terminal width to 65 characters using the variable
MANWIDTH, and the debug output shown me:

Terminal width 65 not within cat page range [80, 80]

Since lots of people do not use 80 char width terminals anymore,
mostly because of "maximize window", I get this function is not
of much use lately, at least not by using Debian default value.
That thing can be configured in /etc/manpath.config if you are
interested in such manipulations.

Well, I don't believe this catman mechanism is of much use
today, most manual pages are rendered in the blink of an eye.
Last time I had to /wait/ more than a seconds for a manual page
to be formatted, that was on a Sun Fire V440.


To summarize, the patched service does a clear the cache 

Why do we have a default activated man-db.timer?

[Tom Bachreier]

System: Testing

Hi!

With the migration of man-db 2.8.5-1 into testing about a month ago I
get bothered once a day with a lot of useless lines in syslog:
> Feb 10 00:00:51 osprey mandb[3543]: Purging old database entries in 
> /usr/share/man...
> Feb 10 00:00:51 osprey mandb[3543]: Processing manual pages under 
> /usr/share/man...
> [...]

Only to come to the same result day after day after day:
> Feb 10 00:00:51 osprey mandb[3543]: 0 man subdirectories contained newer 
> manual pages.
> Feb 10 00:00:51 osprey mandb[3543]: 0 manual pages were added.
> Feb 10 00:00:51 osprey mandb[3543]: 0 stray cats were added.
> Feb 10 00:00:51 osprey mandb[3543]: 0 old database entries were purged.
> Feb 10 00:00:51 osprey systemd[1]: man-db.service: Succeeded.

Thankfully Francois filed a bug earlier:
>

which got fixed upstream:
>

So the messages will go away for me in a couple of days.
Problem solved. :-)

BUT wait...
I was wondering why do we have man-db.timer in the first place?
and
Why is it activated by default?

Manpages in my system changes if I install, remove or update packages.
And I think this will be the case for the majority of debian users.

Shouldn't it be enough to have package install/remove/update event
hooks for updating man-db instead? This renders a daily update job
pretty useless in my opinion.

Do I miss something obvious?

On the other side the man-db packages can come with a default disabled
timer for folk who need it f.ex. who install packages without dpkg/apt
and so on.

What do you think?

Tom