Re: pax and ext2fs

[Walter Alejandro Iglesias]

On Sat May 18 08:50:21 2024 Philip Guenther wrote:
> > So yeah, what's needed is pathconfat(2)** but whether this winding loose 
> > end ("That poor yak.") merits that much code and surface is yet to be 
> > examined deeply.
>
> The fix for this has now been committed, so it'll be in 7.6 and a near
> future snapshot.
>

And you wrote the library!

Philip Guenther at https://austingroupbugs.net/view.php?id=1831 wrote:

   With a fresh cup of coffee, it's 'obvious' the correct action is to
   use pathconfat(AT_FDCWD, path, _PC_TIMESTAMP_RESOLUTION,
   AT_SYMLINK_NOFOLLOW)

   This was touched on in https://austingroupbugs.net/view.php?id=786
   [^] (c.f.  Geoff's comment 2827 from 2015) so maybe I should just
   implement this in OpenBSD and drag everyone else along from there. :)


No yaks for Philip "John Wayne" Guether, only cows. ;-)


>
> Philip Guenther
>
>


Walter

Re: pax and ext2fs

[Walter Alejandro Iglesias]

On Fri May 17 09:50:58 2024 Philip Guenther wrote:
> Sounds like you copied with something like 'cp -p' so the copy has a
> mtime with zero nsecs part, so now they do compare as equal.

This morning I realized that when I copied the symlink from the ext2
drive to my hard disk, cp(1) didn't copy the symlink but the executable
itself.  Reading cp(1) man page I see that the command I should have
used to copy the symlink is 'cp -PR'.

In my case caffeine is affecting negatively, it makes me jump to
conclusions.  Sorry for make you waste your time!

>
>
> > P.S.: I'm curious about the following.  After running the stat command
> > here and there, I found *many* files showing that lack of mtime
> > granularity spread throughout all my system tree (as a side note: this
> > doesn't happen with their ctime and atime.)
>
> The released install tgz files (base75.tgz, etc) use a format where
> the contained files all have simple integer mtimes and tar is invoked
> with the -p option (required for correct permissions on setuid/gid
> files) which makes it also set the mtime on the extracted file to
> match what's in the tar file.
>
> ctime is always set from the local clock when the inode is
> allocated/updated, so no reason for it to always have a zero nsecs.
>
> atime is of course updated from the local clock when you, uh, access them.

Thanks for your explanation!

>
>
> Philip Guenther
>
>

   Walter

Re: pax and ext2fs

[Walter Alejandro Iglesias]

On Thu May 16 09:48:45 2024 Philip Guenther wrote:
> So yeah, what's needed is pathconfat(2)** but whether this winding loose
> end ("That poor yak.") merits that much code and surface is yet to be
> examined deeply.
>
> Philip Guenther
>
>
> ** or lpathconf(2), but pathconfat(2) is better
>

I read what you posted here:

  https://austingroupbugs.net/view.php?id=1831

In the footnote you wrote:

  "(This was encountered when trying to fix a pax implementation's
  handling of timestamp comparison for -u when the target filesystem had
  courser resolution that the source filesystem by using
  pathconf(_PC_TIMESTAMP_RESOLUTION) on the target path to handle the
  loss of high-precision time info...but the symlink pointed to a
  location with high-precision timestamps so it couldn't know to round
  the times when doing the comparison...)"


I did one more experiment.  I removed the offending soft link from my
hard disk, then I copied the backed-up version of the soft link from the
ext2 drive back to my system tree.  Now pax (with your patches) doesn't
insist in re-updating the file, *even after updating the file with
touch(1)*.

The soft link *still* points to a location with high-precision
timestampts, but pax does the right job.

Intuitively this suggests me that there is something more that mtime
precision in this misunderstanding between OpenBSD and ext2 file
systems.  If I copy files using pax from Linux (another *BSD* version of
pax) to that same ext2 drive it works as expected.


  Walter


P.S.: I'm courious about the following.  After running the stat command
here and there, I found *many* files showing that lack of mtime
granularity spread throughout all my system tree (as a side note: this
doesn't happen with their ctime and atime.)

Re: pax and ext2fs

[Walter Alejandro Iglesias]

On Wed May 15 13:04:53 2024 Walter Alejandro Iglesias wrote:
> After more testing I realized that I was wrong my modification doesn't
> solve the problem.
>

Yeah, I also realized that what I did was stupid. :-)

Re: pax and ext2fs

[Walter Alejandro Iglesias]

On Wed May 15 10:24:32 2024 Walter Alejandro Iglesias wrote:
> I get it working but I don't know if what I did is fine.
>
> As I'd told you the problem was ctime (when using -Y), so I added one
> conditional to your diff where it checks only mtime and it works:
>
>
> Index: ar_subs.c
> ===
> RCS file: /cvs/src/bin/pax/ar_subs.c,v
> diff -u -p -r1.51 ar_subs.c
> [...]

After more testing I realized that I was wrong my modification doesn't
solve the problem.

Re: pax and ext2fs

[Walter Alejandro Iglesias]

On Wed May 15 10:20:04 2024 Philip Guenther wrote:
> I think you've managed to hit a spot where the POSIX standard doesn't
> provide a way for a program to find the information it needs to do its job
> correctly.  I've filed a ticket there
>https://austingroupbugs.net/view.php?id=1831
>
> We'll see if my understanding of pathconf() is incorrect or if someone has
> a great idea for how to get around this...
>
>
> Philip Guenther
>

Hi Philip,

I get it working but I don't know if what I did is fine.

As I'd told you the problem was ctime (when using -Y), so I added one
conditional to your diff where it checks only mtime and it works:


Index: ar_subs.c
===
RCS file: /cvs/src/bin/pax/ar_subs.c,v
diff -u -p -r1.51 ar_subs.c
--- ar_subs.c   10 Jul 2023 16:28:33 -  1.51
+++ ar_subs.c   15 May 2024 08:19:08 -
@@ -146,23 +146,61 @@ list(void)
 }
 
 static int
-cmp_file_times(int mtime_flag, int ctime_flag, ARCHD *arcn, struct stat *sbp)
+cmp_file_times(int mtime_flag, int ctime_flag, ARCHD *arcn, const char *path)
 {
struct stat sb;
+   long res;
 
-   if (sbp == NULL) {
-   if (lstat(arcn->name, ) != 0)
-   return (0);
-   sbp = 
+   if (path == NULL)
+   path = arcn->name;
+   if (lstat(path, ) != 0)
+   return (0);
+
+   /*
+* The target (sb) mtime might be rounded down due to the limitations
+* of the FS it's on.  If it's strictly greater or we don't care about
+* mtime, then precision doesn't matter, so check those cases first.
+*/
+   if (ctime_flag && mtime_flag) {
+   if (timespeccmp(>sb.st_mtim, _mtim, <=))
+   return timespeccmp(>sb.st_ctim, _ctim, <=);
+   if (!timespeccmp(>sb.st_ctim, _ctim, <=))
+   return 0;
+   /* <= ctim, but >= mtim */
+   } else if (mtime_flag) {
+   return timespeccmp(>sb.st_mtim, _mtim, <=);
+   } else if (ctime_flag)
+   return timespeccmp(>sb.st_ctim, _ctim, <=);
+   else if (timespeccmp(>sb.st_mtim, _mtim, <=))
+   return 1;
+
+   /*
+* If we got here then the target arcn > sb for mtime *and* that's
+* the deciding factor.  Check whether they're equal after rounding
+* down the arcn mtime to the precision of the target path.
+*/
+   res = pathconf(path, _PC_TIMESTAMP_RESOLUTION);
+   if (res == -1)
+   return 0;
+
+   /* nanosecond resolution?  previous comparisons were accurate */
+   if (res == 1)
+   return 0;
+
+   /* common case: second accuracy */
+   if (res == 10)
+   return arcn->sb.st_mtime <= sb.st_mtime;
+
+   if (res < 10) {
+   struct timespec ts = arcn->sb.st_mtim;
+   ts.tv_nsec = (ts.tv_nsec / res) * res;
+   return timespeccmp(, _mtim, <=);
+   } else {
+   /* not a POSIX compliant FS */
+   res /= 10;
+   return ((arcn->sb.st_mtime / res) * res) <= sb.st_mtime;
+   return arcn->sb.st_mtime <= ((sb.st_mtime / res) * res);
}
-
-   if (ctime_flag && mtime_flag)
-   return (timespeccmp(>sb.st_mtim, >st_mtim, <=) &&
-   timespeccmp(>sb.st_ctim, >st_ctim, <=));
-   else if (ctime_flag)
-   return (timespeccmp(>sb.st_ctim, >st_ctim, <=));
-   else
-   return (timespeccmp(>sb.st_mtim, >st_mtim, <=));
 }
 
 /*
@@ -842,14 +880,12 @@ copy(void)
/*
 * if existing file is same age or newer skip
 */
-   res = lstat(dirbuf, );
-   *dest_pt = '\0';
-
-   if (res == 0) {
+   if (cmp_file_times(uflag, Dflag, arcn, dirbuf)) {
+   *dest_pt = '\0';
ftree_skipped_newer(arcn);
-   if (cmp_file_times(uflag, Dflag, arcn, ))
-   continue;
+   continue;
}
+   *dest_pt = '\0';
}
 
/*

Re: pax and ext2fs

[Walter Alejandro Iglesias]

Hi Philip,

On Tue May 14 19:40:04 2024 Philip Guenther wrote:
> If you like, you could try the following patch to pax to more gracefully
> handle filesystems with time resolution more granular than nanoseconds.

After applying your patch, as I'd done before reporting the issue, I
sycronized my home directory to an external ext2fs drive with the
command showed by the man page:

  $ pax -rw -v -Z -Y source target

This time only one file stays updating again an again, a soft link I
have in my ~/bin folder of /usr/local/bin/prename.  I tried the command
Stuart Henderson taught me in that file:

$ stat -f %Fm /usr/local/bin/prename
1713451867.0

... no sub-second timestamp, like happens when I run the same stat
comand with the files in the ext2fs drive.  I ran stat with other files
under /usr/local, same result, I end noticing that /usr/local is the
only partition mounted with the wxallowed option.

I wish my guessing info will be useful. :-)  Let me know what more I can
do to help.


> The whitespace will presumably be mauled by gmail so use patch's -l option.

Some lines in the diff arrived wrapped but I corrected them and could
apply the patch.  Gmail has an well hidden option :-), if you open your
account from your browser you can configure it to send in plain text.
You have to click in Compose, then in the compose window go to the last
icon in the bottom right (with a vertical ellipis) hovering with your
mouse says "More options", click and you'll see there the "Plain Text"
option, select it and the option stays saved.  I don't use gmail since a
long time, I had to investigate this tired of friends sending me the
HTML copy of all their messages.

>
> Philip Guenther
>
>

-- 
Walter

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Tue May 14 18:11:16 2024 Страхиња Радић wrote:
> Antipatterns are bad. I don't mean the ellipsis in `ls -l ...`. I mean 
> things like
>
>   cat file | grep hello | cat | sed 's/hello/world/g' | cat - > output
>
>   for file in `echo `ls *` `; do echo $file; done
>
>   ls -l | awk '{ print $5 }'  # different things with different ls'es
>   # under different locales, and on 
>   # different systems, with differently
>   # named files
>
> which are something a novice will see and adopt, especially when it is 
> not even communicated as a "sketch", and a seasoned user of shell will 
> just be annoyed with.

This recalls me again the quote of the article you linked:

  "..., unless extreme portability is more important..."

Let's make it short.  Would you guarantee that your shell scripts work
on any unix-like system?


And here I abandon the discussion, needless to say that I appreciate all
your advices.

Greetings!

Re: What software to debugging and analyzing C?

[Walter Alejandro Iglesias]

On Tue May 14 11:40:42 2024 Tomasz Rola wrote:
> I am sure gdb has some merits but for whatever C programs I wrote so
> far, a much more useful debugging technique was putting printf in
> right places and isolate the problem,

I got used to doing this too.  I started doing it intuitively, I'm
self-taught (and I'm certainly not an expert).

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Tue May 14 11:11:33 2024  wrote:
> When `while ... read ...` idiom is used, it is advisable to clear IFS 
> to turn off field splitting, and use -r to avoid interpretation of 
> backslash sequences in the input:
>
>   while IFS= read -r dir; do # ...
>
> Back to parsing the output of ls(1) (also applicable to parsing the 
> output of find(1), or globs), there is an indepth analysis of the 
> problem at [1]. The accepted answer concludes that perhaps shell 
> command language is not the right tool for the job, and a more 
> sophisticated language should be used instead. While I don't agree with 
> the author's choice of Python, any language supporting opendir(3), 
> readdir(3) or equivalent functions will suffice.
>
> [1]: 
> https://unix.stackexchange.com/questions/128985/why-not-parse-ls-and-what-to-do-instead
>

Let's start for what the first answer in that forum put in bold:

   Bourne shell is a bad language.  It should not be used for anything
   complicated, unless extreme portability is more important than any
   other factor (e.g. autoconf).

If you analyze that statement, depending on the case, it may have sense
or not at all.

I learned about the convenience of adding the '-r' option in the "while
read" loop many years ago when I was writing a script to convert roff to
html, the problem aroused with the backslash in roff comments (.\").

When I post an example of a shell script in some forum or mailing list I
post an sketch, assuming others will use it as a example and write
themselves their own solution.  Who won't be able to overcome issues
like the above are those who aren't familiar or trained in that language
in particular.  For me (as I think it should be for any unix user) shell
scripting is mainly the way a "use" the computer, it's not a
"programmers" language, something you use to write whole applications of
the kind "Push this button and relax, I'll do the job".  I have a
hundred of dirty shell scripts in my ~/bin directory that if you examine
them you'll find many dumb errors, but mainly they do the job.  I don't
know anything about python (I don't like it), but I bet that if you
analyze python, or C or Perl, you'll also find inconsistencies you'll
have to workaround as with shell scripting.  Nothing is perfect when you
see it in detail.

Many years ago I posted in some linux forum an example of a shell script
to blacklist IPs in a web-mail server.  My intention was encouraging
users to not follow the MSWin approach, I mean downloading some 3rd
party tool instead of learning what the system already has to offer.
And I remember myself proposing and giving solutions with rsync in that
same forum to someone asking for a mirror capable synchronizing tool.
This, and *learning to do things by yourself* (even if your program isn't
as good the one you download or isn't good at at all), are the
fundamental tendencies I always defend since ARE THE REASON OF EXISTENCE
OF FOSS.  Immediately, some "experts" started to point me "holes" in my
script, when what I posted was just a sketch.  In that case disregarding
shell scripting wasn't useful as an argument since they were proposing
fail2ban.sh (a clear example of using shell scripting for something
complicated. ;-))

Summarizing, my motivation was triggered by the topic of the thread and
the way it was raised by the OP.


-- 
Walter

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Sun May 12 21:50:12 2024 Martin Schröder wrote:
>
>   If a line begins with "- " (dash, space) or "+ " (plus, space),
>   then the type of rule is being explicitly specified as an exclude
>   or an include (respectively).  Any rules without such a prefix are
>   taken to be an include.

I'd read the man page.  What I understand from this paragraph is that
you have to include in the list *all* the files.  How convenient! :-)

>
>
> Coming back to the topic of this thread:

Yeah, because I'm talking about football.


>
> Best
>  Martin
>
>
>

Re: What software to debugging and analyzing C?

[Walter Alejandro Iglesias]

Otto Moerbeek thought me this:

First compile your program with debug symbols (and, conveniently, without
optimization settings.)

  $ DEBUG="-g -O0" make

Then:

  $ MALLOC_OPTIONS=D ktrace -tu 
  $ kdump -u malloc

kdump will though you lines like this:

  0x34f10a4b153   20480  1  20480 addr2line -e /usr/lib/libc.so.97.1 0x4d153
  0x34f10a96470  410576 25  16423 addr2line -e /usr/src/usr.bin/ 0x98470

If you compiled your program with debugging symbols and your program has
some leak, the name of your program will appear in some of those lines.
Then you run that addr2line command and it'll show you in which file and
line the leak is produced.  It will show your errors that valgrind won't.

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Sun May 12 20:58:43 2024 Andreas Kähäri wrote
> With rsync(1):
>
>   rsync -n -aim --delete-excluded \
>   --include-from=list \
>   --include='*/' \
>   --exclude='*' \
>   source/ target
>

I don't understand what your command does exactly.  And this is surely
of everyone interest since, like me, everyone has many files and
directories in $HOME which are not worth to save, as ~/.cache for
example.  If you take a second look to my examples, the one using pax
and the other using rsync inside a loop, both are thought to synchronize
*only* what I put in the list and, as you see, in the list there are
files and directories.  I appreciate that you or any other rsync expert
here show me how to accomplish that (in case it's possible) with rsync
without resorting to a loop as I did.

(Not challenging, I'm asking this as favor.) 


I use a simliar solution to syncronize my $HOME directory to other
machines.  Related to the idea of synchronizing a selection of files and
directories saved in a list, I attempted to lern how rdist(1) works, as
Robert B. Carleton advised me, but I couldn't see much, rdistd(1) core
dumped.

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Sun May 12 13:22:13 2024 Alexis wrote:
> Andreas Kähäri  writes:
> > Well, that's one way to control this trainwreck of a script; 
> > just say
> > that any name containing "inappropriate" characters aren't 
> > allowed!
> >
> > May I ask why you don't simply use rsync(1) (or even 
> > openrsync(1) from
> > the OpenBSD base system)?
>
> i'm not sure why you're addressing this to me, as i'm not the OP.

I guess it's me who Andreas should address this question to, right?

I gave a dirty example to someone who mentioned pax to the OP.  Just
playing and lerning. :-)

I've been using rsync since ever, but, first, I don't think rsync is
bulletproof either and, second, making a backup with pax is faster and
in some cases simpler.  If you don't want to delete files on the target
you don't need to do scripting at all.

Let's take the example I put in my first message.  With rsync, you'll
have to do something like this:

~/backup_list

# backup_list
.Xdefaults
.kshrc
.nexrc
.profile
.calendar/
.config/feh/
.config/fontconfig/
.config/gtk-3.0/gtk.css
Documents/
Pictures/
[...]
---

files=$(egrep -v "^$|^#" ~/backup_list)

for i in $files ; do
rsync -av --delete --mkpath $HOME/$i $device/$user/$i
done


But openrsync doesn't have a '--mkpath' option, I let Andreas to think
the solution. :-)


>
> That said, yes, minimising the extent to which certain non-'word' 
> characters (i.e. roughly the POSIX 'alnum' class as described in 
> re_format(7)) _can_ make it easier to programatically do certain 
> tasks which are restricted by the long and messy history of C and 
> Unix development. Given that i've been using computers for a few 
> decades, i still instinctively don't use spaces in filenames, even 
> though they're very much allowed. But of course, that's not what 
> most of the world does, and this is an example of trying to work 
> out what the best tradeoffs might be when dealing with the 
> messiness of the real world.

I overlooked this in my example because I *never* use spaces, UTF-8 or
any special characters to name my file names.  Lately, I finally
persuaded my wife to use Linux, after decades of having to use Windows.
Even when I educated her in this matter she has clients who send her
files named with any kind of crap, so taking care of this issue is still
convenient.


>
>
> Alexis.
>
>
>

-- 
Walter

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Sun May 12 11:40:05 2024 tux2bsd wrote
> Hi Walter
>
> mktemp makes temporary unique filenames like this:
>
> delete_list=$(mktemp)
> source_list=$(mktemp) 
> target_list=$(mktemp) 
> # Do your code. If you want to keep something you do
> # that appropriately then:
> rm $delete_list $source_list $target_list
>
>

This version can deal with files with special characters and spaces:

# Remove files from target directory
delete_list=$(mktemp -t delete.XX) || exit 1
source_list=$(mktemp -t source.XX) || exit 1
target_list=$(mktemp -t target.XX) || exit 1

dirs=$(echo "$files" | grep '/$')

cd && find $dirs | sort | uniq > $source_list
cd "$target" && find $dirs | sort | uniq > $target_list
diff $source_list $target_list |
grep '^> ' | sed 's#^> #'$target'/#' > $delete_list

cd &&
while read line; do
echo "delete $line"
rm "$line"
done < $delete_list

# Clean
rm $source_list $target_list $delete_list

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Sun May 12 10:07:30 2024 Страхиња Радић wrote:
> A few notes:
> 
> - You don't need a backslash after a pipe (|) or a list operator (||
>   and &&) - a line ending with a pipe is an incomplete pipeline. So 
>   (with added quoting):
> 
>   diff "$source_list" "$target_list" |
>   awk '/^> / { print "'"$target"'/" $NF }' > "$delete_list"

I know, just fingers habit. :-)

>
>   As an example for a list operator, the second line beginning with cd
>   could also be written as:
> 
>   cd "$target" &&
>   find "$dirs" | sort | uniq > "$target_list"
> 
>   This works even when entering commands interactively from the command 
>   line.
> 
> - Before the `rm -rf` line, a useless use of cat[1]:
> 
>   sed 's/^/delete /' "$delete_list"
> 
> - The xargs is unnecessary in `rm -rf $(cat $delete_list | xargs)`; 
>   BTW, that line is vulnerable to weird pathnames (for example, 
>   those including spaces, line feeds and special characters).
> 

What about the following, better?

-
# Remove files from target directory
date=$(date +%H%M%S)
delete_list=/tmp/delete_$date
source_list=/tmp/source_$date
target_list=/tmp/target_$date

dirs=$(echo "$files" | grep '/$')

cd && find $dirs | sort | uniq > $source_list
cd "$target" && find $dirs | sort | uniq > $target_list
diff $source_list $target_list |
 awk '/^> / { print "'$target'/" $NF }' > $delete_list

cd &&
if [ -s $delete_list ]; then
echo "Deleting on ${target}:"
rm -vrf $(cat $delete_list)
fi

# Clean
rm $source_list $target_list $delete_list
-


Thanks for your recomendations!


-- 
Walter

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Sat May 11 20:20:04 2024 "Robert B. Carleton" wrote:
> Another tool you might want to take a look at is rdist(1). It's limited
> in some ways, but is a native capability to OpenBSD. It has a long
> history.
>

I've never used rdist(1) either, I will learn about it.  Thanks Robert
for mention it to me!

With unix I always feel like I'm just starting out, blissfully ignorant,
(I was about to also say "young" but that would be an exaggeration :-)).

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Fri May 10 08:36:50 2024 Walter Alejandro Iglesias wrote
> Then I do something like this (simplified for clartiy):
>
> [...]
>
> dirs=$(echo "$files" | grep '/$')
>
> for i in $dirs ; do
>   find $source/$i | sed 's#'$source'##' | sort | uniq > $source_list
>   find $target/$i | sed 's#'$target'##' | sort | uniq > $target_list
>   diff $source_list $target_list |\
>awk '/^> / { print "'$target'" $NF }' >> $delete_list
> done
>
> cat $delete_list | sed 's/^/delete /'
> rm -rf $(cat $delete_list | xargs)
>
> rm $source_list $target_list $delete_list
> 
>


Today I realized that the loop above is not necesary:

---
dirs=$(echo "$files" | grep '/$')

cd && find $dirs | sort | uniq > $source_list
cd $target && find $dirs | sort | uniq > $target_list
diff $source_list $target_list |\
 awk '/^> / { print "'$target'/" $NF }' > $delete_list

cat $delete_list | sed 's/^/delete /'
rm -rf $(cat $delete_list | xargs)

# Clean
rm $source_list $target_list $delete_list

Re: Favorite configuration and system replication tools?

[Walter Alejandro Iglesias]

On Fri May 10 08:16:32 2024 "Robert B. Carleton"  wrote:
> I'm going to try using pax(1) in copy mode (-rw) as an alternative to
> rsync and cpio when it's local filesystems. I hadn't considered that
> until recently.

This is my dirty solution to add pax a "delete on target" functionality.

I save the list of the files and directories I want to back up to a file
(starting from my $HOME).  Notice that directories end with a slash (to
facilitate filtering with grep in the script.)


# backup_list
.Xdefaults
.kshrc
.nexrc
.profile
.calendar/
.config/feh/
.config/fontconfig/
.config/gtk-3.0/gtk.css
Documents/
Pictures/
[...]
---


Then I do something like this (simplified for clartiy):

--
backup_list=/path/to/backup_list
source=$HOME
target=$device
files=$(egrep -v "^$|^#" $backup_list)

pax -rw -v -Z -Y $files $target

# Delete files not present in source from target directory
date=$(date +%H%M%S)
delete_list=/tmp/delete_$date
source_list=/tmp/source_$date
target_list=/tmp/target_$date

dirs=$(echo "$files" | grep '/$')

for i in $dirs ; do
find $source/$i | sed 's#'$source'##' | sort | uniq > $source_list
find $target/$i | sed 's#'$target'##' | sort | uniq > $target_list
diff $source_list $target_list |\
 awk '/^> / { print "'$target'" $NF }' >> $delete_list
done

cat $delete_list | sed 's/^/delete /'
rm -rf $(cat $delete_list | xargs)

rm $source_list $target_list $delete_list



-- 
Walter

Using my patched version of OpenBSD mail(1).
https://en.roquesor.com/Downloads/mail_patches.tar.gz

Re: pax and ext2fs (update)

[Walter Alejandro Iglesias]

Removing the inode check (-Y option) files are updated correctly to ext2fs.
So the command would be:

  $ pax -rw -v -Z $files $target

So, it's something with the inode check what doesn't work with ext2fs.

Re: pax and ext2fs

[Walter Alejandro Iglesias]

On Thu, 2 May 2024 12:03:10, Stuart Henderson wrote
> I don't have a suitable filesystem handy to test, but does OpenBSD's
> implementation of ext2fs support sub-second timestamps?
>
> stat -f %Fm $filename
>
> If not, that's a probable explanation for the difference in behaviour.
> You could probably confirm by forcing timestamps with no nanosecond
> components, e.g. touch -t mmddhhmm.ss $filename, or copy to ext2fs
> and back again.

$ doas mount -t ext2fs /dev/sd0i /mnt
$ touch ~/test.txt
$ cp ~/test.txt /mnt
$ stat -f %Fm /mnt/test.txt
1714657214.0
$ cp ~/test.txt /mnt
$ stat -f %Fm /mnt/test.txt
1714657409.0
Â~m

Re: pax and ext2fs

[Walter Alejandro Iglesias]

I expect from that command no more and no less than what is explained in
the man page:

Update (and list) only those files in the destination directory
/backup which are older (less recent inode change or file
modification times) than files with the same name found in the source
file tree home:

$ pax -r -w -v -Y -Z home /backup

While that works as explained when copying to a ffs drive, running the
same command again and again to a ext2fs target *all* files are copied
again, even those already updated, what suggests me that pax(1) fails to
get ext2fs timestamps right.  But this is asking to much to OpenBSD,
right?  Linux don't even support ffs.  That's why I didn't put much care
in reporting this issue or sending it to bugs@.  Besides, in general, to
backup my files, I don't trust synchronizing files (for which I use
rsync) to foreign file systems, in this case I save a *.tag.gz to the
drive.

By the way, currently I'm not using OpenBSD in any production system but
in my desktop and just for fun.  So, most of the time when I post
something here I'm not asking for help but trying to contribute to the
project.  I appreciate your concern about my luck anyways.


-- 
Walter

Using my patched version of OpenBSD mail(1).
https://en.roquesor.com/Downloads/mail_patches.tar.gz

pax and ext2fs

[Walter Alejandro Iglesias]

Hello,

I'd never used pax(1), reading the man page I found this command can be
used to make a backup:

  $ pax -r -w -v -Y -Z home /backup

Faster than using rsync indeed, but it seems that the -Y and -Z options
don't work with ext2fs?


-- 
Walter

A new version of fmt enhanced for *roff files

[Walter Alejandro Iglesias]

Hello everyone,

I wrote my own version of fmt, with some enhancements and a new feature
to break lines in *roff files.

  https://en.roquesor.com/Downloads/fmtroff.c

In the head comment is explained why I reinvented the wheel. :-)

I guess someone could find it useful.


-- 
Walter

init ran syspatch(8) after upgrading to latest snapshot

[Walter Alejandro Iglesias]

I'm posting this here since I'm not sure if it's a bug or something I
did wrong.  Today I upgraded to the latest snapshot and while booting to
the new system init ran syspatch(8).  I can't figure out why.


dmesg:
OpenBSD 7.2 (GENERIC.MP) #720: Sun Sep 11 15:41:58 MDT 2022
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4135260160 (3943MB)
avail mem = 3992539136 (3807MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xeb920 (70 entries)
bios0: vendor Hewlett-Packard version "786E1 v01.16" date 08/17/2011
bios0: Hewlett-Packard HP Compaq dc7700 Convertible Minitower
acpi0 at bios0: ACPI 1.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC ASF! MCFG TCPA SLIC HPET
acpi0: wakeup devices PCI0(S4) COM1(S4) PEG1(S4) IGBE(S4) PCX1(S4) PCX2(S4) 
HUB_(S4) USB1(S3) USB2(S3) USB3(S3) USB4(S3) USB5(S3) EUS1(S3) EUS2(S3) PBTN(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 CPU 4300 @ 1.80GHz, 1795.56 MHz, 06-0f-02
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF,SENSOR,MELTDOWN
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 2MB 64b/line 
8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 199MHz
cpu0: mwait min=64, max=64, C-substates=0.2, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM)2 CPU 4300 @ 1.80GHz, 1795.51 MHz, 06-0f-02
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF,SENSOR,MELTDOWN
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 2MB 64b/line 
8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins, remapped
acpimcfg0 at acpi0
acpimcfg0: addr 0xf400, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (PEG1)
acpiprt2 at acpi0: bus 32 (PCX1)
acpiprt3 at acpi0: bus -1 (PCX2)
acpiprt4 at acpi0: bus 7 (HUB_)
acpipci0 at acpi0 PCI0: 0x 0x0011 0x0001
acpicmos0 at acpi0
com0 at acpi0 COM1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
"PNP0003" at acpi0 not configured
tpm0 at acpi0 TPM_ 1.2 (TIS) addr 0x4e/0x2, device 0x rev 0xff
acpibtn0 at acpi0: PBTN
"PNP0C14" at acpi0 not configured
acpicpu0 at acpi0: C1(@1 halt!), PSS
acpicpu1 at acpi0: C1(@1 halt!), PSS
cpu0: Enhanced SpeedStep 1795 MHz: speeds: 1800, 1200 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82Q965 Host" rev 0x02
inteldrm0 at pci0 dev 2 function 0 "Intel 82Q965 Video" rev 0x02
drm0 at inteldrm0
intagp0 at inteldrm0
agp0 at intagp0: aperture at 0xe000, size 0x1000
inteldrm0: apic 1 int 16, I965G, gen 4
"Intel 82Q965 HECI" rev 0x02 at pci0 dev 3 function 0 not configured
em0 at pci0 dev 25 function 0 "Intel ICH8 IGP AMT" rev 0x02: apic 1 int 19, 
address 00:0f:fe:77:4f:df
uhci0 at pci0 dev 26 function 0 "Intel 82801H USB" rev 0x02: apic 1 int 20
uhci1 at pci0 dev 26 function 1 "Intel 82801H USB" rev 0x02: apic 1 int 21
ehci0 at pci0 dev 26 function 7 "Intel 82801H USB" rev 0x02: apic 1 int 22
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 
addr 1
azalia0 at pci0 dev 27 function 0 "Intel 82801H HD Audio" rev 0x02: apic 1 int 
21
azalia0: codecs: Realtek ALC262
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 82801H PCIE" rev 0x02
pci1 at ppb0 bus 32
uhci2 at pci0 dev 29 function 0 "Intel 82801H USB" rev 0x02: apic 1 int 20
uhci3 at pci0 dev 29 function 1 "Intel 82801H USB" rev 0x02: apic 1 int 21
ehci1 at pci0 dev 29 function 7 "Intel 82801H USB" rev 0x02: apic 1 int 20
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 
addr 1
ppb1 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xf2
pci2 at ppb1 bus 7
pcib0 at pci0 dev 31 function 0 "Intel 82801HO LPC" rev 0x02
pciide0 at pci0 dev 31 function 2 "Intel 82801H SATA" rev 0x02: DMA, channel 0 
configured to native-PCI, channel 1 configured to native-PCI
pciide0: using apic 1 int 18 for native-PCI interrupt
wd0 at pciide0 channel 0 drive 0: 
wd0: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 
addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 
addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 

em(4) issue I reported time ago seems to be fixed now

[Walter Alejandro Iglesias]

Today I burned the latest snapshot in a USB pen-drive and booted it
in a HP desktop of mine where happened this:

https://marc.info/?l=openbsd-bugs=156682947025229=2

It seems to be fixed, the card is now recognized out of the box as
1000baseTX as expected. :-)  (I didn't change anything in my LAN,
the machine is connected to the same cable to the same router as
it was at that time.)

Does someone know or have a clue which change fixed this issue?


Walter

Re: Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

In article  Peter Nicolai 
Mathias Hansteen  wrote:
> It is a possibly desirable feature, but I an not aware whether any of the 
> currently capable developers are considering putting in the work to implement 
> it.
> 

Let me finish the idea, not with the intention to pressure developers
asking for features but to share my experience and thoughts about the
issue.

I've also been publishing (long) blacklists in my website as you do.  As
an experiment I didn't expire any until recently when, as I explained,
they reached the hard limit in memory (200).  And, as I suggested,
right before I expired addresses old spam, recognizable by the format,
appeared again.

The problem is most system administrators out there do very little.  If
you were getting spam or attacks from some IP, even if you report the
issue to the respective whois abuse@ address, chances are attacks from
that IP won't stop next week, nor even next month.

So, in general terms, I would refrain as much as possible from hurry to
expiring addresses.  Just my opinion.

Re: Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

Hello Peter,

In article  Peter Nicolai 
Mathias Hansteen  wrote:
> > 28. mai 2020 kl. 19:09 skrev Bruno Flueckiger :
> > 
> > 
> > You can save the list of IPs in a table and reload it after a reboot as
> > described here: https://www.bsdhowto.ch/savepftables.html
> 
> 
> I have a similar setup at bsdly.net , only I dump the 
> tables to file and run expiry via a cron job that runs twice an hour - the 
> writeup at 
> https://bsdly.blogspot.com/2017/04/forcing-password-gropers-through.html 
>  
> has most of the useful info and some related wrinkles.
> 

Could you summarize here which part of these articles of yours answer my
original question, please?

For example, this list you share (linked in your article):

  https://home.nuug.no/~peter/pop3gropers_full.txt

It would be great to be able to do the following before and after a
reboot respectivelly:

  # pfctl -t smtp -vT show > file   # (notice the verbose option)
  # pfctl -t smtp -T replace -f file

But we know that doesn't work.


> All the best,
> 
> 
> —
> Peter N. M. Hansteen, member of the first RFC 1149 implementation team
> http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
> "Remember to set the evil bit on all malicious network traffic"
> delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
> 
> 

Re: Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

In article <20200528165448.ga22...@flueckiger.lan> Bruno Flueckiger 
 wrote:
> On 26.05., Walter Alejandro Iglesias wrote:
> > I understand that this command:
> >
> >   # pfctl -t spam -T expire 
> >
> > Takes in care the "Cleared" date:
> >
> >   # pfctl -t spam -vT show
> >  ___.___.22.65
> >   Cleared: Mon May 25 16:10:22 2020
> >  ___.___.167.62
> >   Cleared: Mon May 25 16:10:22 2020
> >   [...]
> >
> > Is there a way to save and restore tables metadata after a reboot
> > preserving those dates?
> >
> 
> You can save the list of IPs in a table and reload it after a reboot as
> described here: https://www.bsdhowto.ch/savepftables.html

Nice website. ;-)

> 
> As there is no way to save the dates the date for each IP will be set to
> the current date and time when load happens.

The interesting point and the reason of my concern is to choose a
convenient "expire time."  With mail is problematic but with ssh, since
I know exactly whom I want to allow external access (just me,) I let
them accumulate.  I block ssh attackers in the ssh port only, people
sharing those addresses are not affected.  So, I thought, the only
concern in the ssh case was how much a big number of entries could
affect pf performance, till at some point my tables reached the memory
hard limit and I had to remove IPs arbitrarily. :-)

In summary, pfctl expire command does nothing after a reboot.  Then you
have two options:

  - To use a (cron) expire time significantly lower than the desirable.

  - To expire entries when your tables are about to reach the memory
hard limit.

In both cases you'll probably suffer spam again from IPs that were
already blocked.


> 
> Cheers,
> Bruno
> 
> 

Walter

Re: About pf max-src-conn-rate

[Walter Alejandro Iglesias]

On Thu, May 28, 2020 at 12:06:18PM +0200, Marko Cupać wrote:
> On 2020-05-27 14:27, Walter Alejandro Iglesias wrote:
> > Another question about pf.
> > 
> > Perhaps I don't fully understand how connection rate is calculated.
> > 
> > The following line in /etc/pf.conf:
> > 
> >   pass in log inet proto tcp to any port { smtp smtps } synproxy state 
> > \
> > (max-src-conn-rate 5/30, overload  flush global)
> > 
> > Shouldn't avoid this happen?
> > 
> > In /var/log/maillog
> > ...
> > A total of *323* connections from the same IP at less than a 1/4 second
> > interval during more than four minutes.
> 
> If I'm not mistaken (someone please correct me if I'm wrong), 323 
> connections in maillog is not the same as 323 tcp connections. You can 
> send 323 smtp commands in single tcp session.

That's been my suspicion so far, that's why I didn't ask this question
here before.  I have to study how smtp connections work, if you're
right, then that's what's happening.

> 
> Perhaps you should look into https://man.openbsd.org/spamd to achieve 
> your goal.

What I do is enough to keep mail spam under control in my case.  My
doubt was mostly technical.


> 
> -- 
> Before enlightenment - chop wood, draw water.
> After  enlightenment - chop wood, draw water.
> 
> Marko Cupać
> https://www.mimar.rs/


Thanks Marko!

Re: About pf max-src-conn-rate

[Walter Alejandro Iglesias]

Brian Brombacher wrote:

> Keep in mind operations using pfctl such as reloading rule set or table
> from file, any IP’s caught in the smtp table by the max-src-conn-rate
> will be flushed de pending on your command line.

> Every hour I scrape logs for AUTH failures and add them to a pfctl
> table using pfctl -t table_name -T add ip_address.

These are the pfctl commands I use in the cron script.  In this order,
no more and no less:

  # Expire old entries
  pfctl -q -t smtp -T expire $(expr $days \* 86400)

  # Add new entries to table
  pfctl -q -t smtp -T add -f /tmp/newaddresses

  # Save list to file
  pfctl -q -t smtp -T show > /path/to/smtp.txt

(By the way, the 'expire' command is the reson of my first question in
the "Restore pf tables metadata after a reboot" thread.)

I'll do the test I mentioned before, I'll add a provisional table
affected only by the max-src-conn-rate.


Walter

Re: About pf max-src-conn-rate

[Walter Alejandro Iglesias]

Hello Brian,

On Wed, May 27, 2020 at 02:35:46PM -0400, Brian Brombacher wrote:
> What do you do with  table in other rules?  If you’re doing nothing, 
> you need to do something like block additional connections, or adjust the 
> pass rule to include from ! 

You're right.  I forgot to mention I have these lines before:

  table  persist file "/path/to/smtp.txt"
  block in log quick inet proto tcp from  to any port { smtp smtps }

> 
> Run: pfctl -t smtp -T show
> 
> Does it show the offending IP?  If so, the rule worked as you defined it.
> 
> 

I run a cron script that parses my log files and also add the offending
IPs to that table.  To be sure the max-src-conn-rate adds those IPs to
the table I'll have to create an alternative table just to test.

About pf max-src-conn-rate

[Walter Alejandro Iglesias]

Another question about pf.

Perhaps I don't fully understand how connection rate is calculated.

The following line in /etc/pf.conf:

  pass in log inet proto tcp to any port { smtp smtps } synproxy state \
(max-src-conn-rate 5/30, overload  flush global)

Shouldn't avoid this happen?

In /var/log/maillog

May 27 10:55:05 server smtpd[30272]: 1a931fba4746f485 smtp connected 
address=192.119.68.113 host=hwsrv-733438.hostwindsdns.com
May 27 10:55:06 server smtpd[30272]: 1a931fba4746f485 smtp failed-command 
command="RCPT TO:" result="550 Invalid recipient: 
"
May 27 10:55:06 server smtpd[30272]: 1a931fba4746f485 smtp disconnected 
reason=disconnect
May 27 10:55:06 server smtpd[30272]: 1a931fbbc5c841e4 smtp connected 
address=192.119.68.113 host=hwsrv-733438.hostwindsdns.com
May 27 10:55:06 server smtpd[30272]: 1a931fbbc5c841e4 smtp failed-command 
command="RCPT TO:" result="550 Invalid recipient: 
"
May 27 10:55:07 server smtpd[30272]: 1a931fbbc5c841e4 smtp disconnected 
reason=disconnect
May 27 10:55:07 server smtpd[30272]: 1a931fbc9f586ee6 smtp connected 
address=192.119.68.113 host=hwsrv-733438.hostwindsdns.com
May 27 10:55:07 server smtpd[30272]: 1a931fbc9f586ee6 smtp failed-command 
command="RCPT TO:" result="550 Invalid recipient: 
"
May 27 10:55:07 server smtpd[30272]: 1a931fbc9f586ee6 smtp disconnected 
reason=disconnect
May 27 10:55:07 server smtpd[30272]: 1a931fbdf6b23f59 smtp connected 
address=192.119.68.113 host=hwsrv-733438.hostwindsdns.com

[...] Complete here with 311 entries with the same time interval. 

May 27 10:59:11 server smtpd[30272]: 1a9320f8f8726fab smtp disconnected 
reason=disconnect
May 27 10:59:11 server smtpd[30272]: 1a9320f9e3e281ab smtp connected 
address=192.119.68.113 host=hwsrv-733438.hostwindsdns.com
May 27 10:59:11 server smtpd[30272]: 1a9320f9e3e281ab smtp failed-command 
command="RCPT TO:" result="550 Invalid recipient: 
"
May 27 10:59:12 server smtpd[30272]: 1a9320f9e3e281ab smtp disconnected 
reason=disconnect
May 27 10:59:12 server smtpd[30272]: 1a9320fa851b3e31 smtp connected 
address=192.119.68.113 host=hwsrv-733438.hostwindsdns.com
May 27 10:59:12 server smtpd[30272]: 1a9320fa851b3e31 smtp failed-command 
command="RCPT TO:" result="550 Invalid recipient: 
"
May 27 10:59:12 server smtpd[30272]: 1a9320fa851b3e31 smtp disconnected 
reason=disconnect
May 27 10:59:13 server smtpd[30272]: 1a9320fbe3f04434 smtp connected 
address=192.119.68.113 host=hwsrv-733438.hostwindsdns.com
May 27 10:59:13 server smtpd[30272]: 1a9320fbe3f04434 smtp failed-command 
command="RCPT TO:" result="550 Invalid recipient: 
"
May 27 10:59:13 server smtpd[30272]: 1a9320fbe3f04434 smtp disconnected 
reason=disconnect
May 27 10:59:13 server smtpd[30272]: 1a9320fc4f172f88 smtp connected 
address=192.119.68.113 host=hwsrv-733438.hostwindsdns.com
May 27 10:59:14 server smtpd[30272]: 1a9320fc4f172f88 smtp failed-command 
command="RCPT TO:" result="550 Invalid recipient: 
"
--

A total of *323* connections from the same IP at less than a 1/4 second
interval during more than four minutes.

Re: Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

On Tue, May 26, 2020 at 11:25:21PM +0200, Anders Andersson wrote:
> On Tue, May 26, 2020 at 2:14 PM Walter Alejandro Iglesias
>  wrote:
> >
> > I understand that this command:
> >
> >   # pfctl -t spam -T expire 
> >
> > Takes in care the "Cleared" date:
> >
> >   # pfctl -t spam -vT show
> >  ___.___.22.65
> >   Cleared: Mon May 25 16:10:22 2020
> >  ___.___.167.62
> >   Cleared: Mon May 25 16:10:22 2020
> >   [...]
> >
> > Is there a way to save and restore tables metadata after a reboot
> > preserving those dates?
> 
> Isn't this what pfctl -S and -L does?

I *guess* what pfctrl -S does is to save in a file the same you see in
'pfctl -s states' output but in binary format.

Restore pf tables metadata after a reboot

[Walter Alejandro Iglesias]

I understand that this command:

  # pfctl -t spam -T expire 

Takes in care the "Cleared" date:

  # pfctl -t spam -vT show
 ___.___.22.65
  Cleared: Mon May 25 16:10:22 2020
 ___.___.167.62
  Cleared: Mon May 25 16:10:22 2020
  [...]

Is there a way to save and restore tables metadata after a reboot
preserving those dates?

Re: Useful cwm patch [was: When will be created a great desktop experience for OpenBSD?]

[Walter Alejandro Iglesias]

In article <20200415193758.csp3wtf4hnhdc...@gmx.com> Dumitru Moldovan 
 wrote:
> On Tue, Oct 29, 2019 at 10:43:26AM +0100, Walter Alejandro Iglesias wrote:
> >This second one is still pending (no response from the maintainer so
> >far):
> >
> >  https://marc.info/?l=openbsd-tech=155931484124288=2
> 
> Apologies for resurrecting a dead and buried thread, but this second
> patch is actually really useful.  Have tested it for a few months as a
> single patch to my 6.6 cwm, it works so good I actually forgot about it.
> 
> CC'ing cwm maintainer in the hope he'll consider it.  Thanks!
> 
> 

Let me add that *this is a bug*, not a feature.  A bug that affects
basic functionality.


Walter

Re: When will be created a great desktop experience for OpenBSD?

[Walter Alejandro Iglesias]

In article <20191028083820.ga43...@nausicaa.home> Marc Espie  
wrote:
> On Fri, Oct 25, 2019 at 05:35:27PM +, flauenroth wrote:
> > Apparently not just theo is using fvwm after all. :) 
> 
> Considering all the people using it, it would be great if someone were to
> look at the enhancements of fvwm2 (wrong license, so not base) and backport
> some of these to our elderly fvwm.
> 
> Specifically, fvwm in base does NOT deal well with multi-screen setups, among
> other things.  It's missing all kinds of extensions that the X server provides
> these days.
> 
> Very much less than perfect experience.
> 
> I have fvwm2 from ports on every machine that runs OpenBSD. No choice about
> that.
> 
> (and I stick with fvwm* because the configuration options for mixing keyboard
> keys with mouse behavior do NOT exist anywhere else)
> 
> 

Another long term fvwm2 user here.

I move my hands off the keyboard (to reach arrows, Pg*, Home, End, keys
or the mouse) only when I'm forced to.  That's why the first feature I
test in a window manager is its switch focus behavior from keyboard.  I
usually bind this function to the popular Alt-Tab.

I tested and used lots of window managers for X, the only one that let
me do what I want is fvwm2, this is why I've got stuck with it (13
years ago and still counting) even when I don't agree with the insanity
that means to have to learn a whole programming language to configure a
window manager.

With the fvwm in base the first problem I bumped against to is that when
switching from firefox to xterm using the keyboard half of the time the
xterm doesn't get the focus.  If you modify the WarpToWindow coordinates
in the WindowList function to bigger values, for example making the
pointer go to the center of the window like cwm does:

DestroyFunc WindowListFunc
AddToFunc WindowListFunc
+ "I" WindowId $0 Iconify -1
+ "I" WindowId $0 FlipFocus
+ "I" WindowId $0 Raise
+ "I" WindowId $0 WarpToWindow 50 50  # <- Here

each time you cycle windows from the keyboard it takes fvwm two seconds
thinking, showing a dot by pointer (what generaly happens to fvwm with
buggy functions,) before doing the switch.  It suffice to say you'll
have a smoother experience with last Robert Nation's version or twm(1).

I like cwm(1) but it's still a bit green and isn't getting enough
attention, I had to insist to get this first patch committed:

  https://marc.info/?l=openbsd-tech=149182817427598=2

This second one is still pending (no response from the maintainer so
far):

  https://marc.info/?l=openbsd-tech=155931484124288=2


Walter

Re: Erratically losing connection on usb port from UPS unit

[Walter Alejandro Iglesias]

Hi Boudewijn,

In article  Boudewijn Dijkstra 
 wrote:
> Op Wed, 21 Aug 2019 15:06:17 +0200 schreef Walter Alejandro Iglesias
> :
> > [...]
> >
> > It has been working for days (with and without NUT) apparently without
> > problems except for three times in which the usb signal from the ups got
> > cut, apparently for no reason.  I get this message in console:
> >
> >   upd0 detached
> >   uhidev0 detached
> 
> I have a similar recurring message, also very erratically, mean period is
> about a month:
> 
> ugen0 detached
> ugen0 at uhub1 port 2 "American Power Conversion Smart-UPS 1000 FW:UPS
> 09.2 / ID=18" rev 2.00/1.06 addr 2
> 
> > While using NUT I got "stale" messages from upsmon when that happened.
> 
> I'm using apcupsd, I also get messages for these occurrences.
> 
> 

I don't know if also in your case but in mine, once that happens, the
connection doesn't get back, what means, from then on, the ups stays
unmonitored.

Another problem I bumped into (this is my first experience with a UPS
unit) is before purchasing this device I'd assumed that to wake up the
machine once the AC was back would suffice activating the "auto power on
when AC is plugged" BIOS option.  Wrong. :-)  Then I tried a router with
dd-wrt software that comes with a wol daemon, it didn't work either.
Routing a wol packet from the internet (adding static arp entries in the
router as they teach in some forums) wasn't either possible.

What makes me conclude: if you can't rely on the machine will be shut
down properly and it's so difficult (automatically or remotely), to get
the machine powered back, what in the Hell a UPS is useful for?

Re: Erratically losing connection on usb port from UPS unit

[Walter Alejandro Iglesias]

On Wed, Aug 21, 2019 at 03:06:17PM +0200, Walter Alejandro Iglesias wrote:
> On the Internet some NUT users mention issues with libusb.  There is an
> old tutorial about using NUT in OpenBSD that advices to install
> libusb-compat but, given the current nut package doesn't install it as a
> dependency I assumed it's not longer necessary.

Here I said something wrong, nut *does* install libusb-compat as a
dependency.

Erratically losing connection on usb port from UPS unit

[Walter Alejandro Iglesias]

Hello everyone,

Weeks ago I purchased a UPS unit for my home server.  It's attached to
the machine via a usb port:

  uhidev0 at uhub3 port 2 configuration 1 interface 0 "EATON Eaton 3S" rev 
2.00/1.00 addr 2
  uhidev0: iclass 3/0, 32 report ids
  upd0 at uhidev0


At first I used NUT from packages but once I became aware of upd(4)
already gives me the needed info:

  hw.sensors.upd0.indicator4=On (ACPresent), OK
  hw.sensors.upd0.percent0=100.00% (RemainingCapacity), OK

... I simply run a shell script from cron to halt the machine when the
battery is low.

It has been working for days (with and without NUT) apparently without
problems except for three times in which the usb signal from the ups got
cut, apparently for no reason.  I get this message in console:

  upd0 detached
  uhidev0 detached

While using NUT I got "stale" messages from upsmon when that happened.


What I Investigated So Far
--

NUT usbhid-ups driver man page mentions some workaround:

  https://networkupstools.org/docs/man/usbhid-ups.html

  pollfreq=num

Set polling frequency, in seconds, to reduce the USB data
flow. Between two polling requests, the driver will wait for
interrupts (aka UPS notifications), which are data changes returned
by the UPS by itself. This mechanism allow to avoid or reduce
staleness message, due to the UPS being temporarily overloaded with
too much polling requests. The default value is 30 (in seconds).

But, since the issue happens erratically, so far I couldn't realize if
that solves the problem (if it does, it would be possible to teach
upd(4) to do something alike?)

On the Internet some NUT users mention issues with libusb.  There is an
old tutorial about using NUT in OpenBSD that advices to install
libusb-compat but, given the current nut package doesn't install it as a
dependency I assumed it's not longer necessary.  There is an item in
libusb-compat package info that explains:

  $ pkg_info libusb-compat
  [...]
   4. Some libusb-0.1 users may have implemented I/O cancellation by
  running transfers in their own threads and simply killing the thread
  when they don't want to do the transfer any more. This is bad
  programming practice for obvious reasons, and this lack of
  functionality was one of the primary drivers for libusb-1.0
  development. With libusb-1.0 or libusb-compat-0.1 backed by
  libusb-1.0, forcefully killing threads in this way is likely to
  cause all libusb I/O to halt. Instead, port your application to use
  libusb-1.0's asynchronous transfer API, which supports transfer
  cancellation.

Is this related?

If someone knows where the problem is I'd appreciate any advice.


Walter

Re: ampd(8) -Z option

[Walter Alejandro Iglesias]

Jan Stary wrote:
> Eventualy it does suspend, but much later than
> when the battery life goes below the specified value.

Doing more testing I noticed the same.  For example, with these options:

 # apmd -d -t 60 -z 

It took *5* minutes to suspend the machine.  In my first tests I passed
bigger values to -t (120 seconds) what made me think -Z was ignored.

Not a dev here but I try to learn when I have time.  I took a look to
/src/usr.sbin/apmd/ampd.c, I see there is a loop (line 507) where the
limit passed to -t is checked, then if AC is not connected the
autoaction is executed.  Time seems to be handled with a system call
(kevent), that's beyond my basic C knowledge. :-)  Perhaps some other
checking in this loop add delays, just guessing.


Walter

Re: ampd(8) -Z option

[Walter Alejandro Iglesias]

Hi Edgar,

On Sun, Aug 04, 2019 at 11:43:19AM -0500, Edgar Pettijohn wrote:
> On Sun, Aug 04, 2019 at 05:33:41PM +0200, Walter Alejandro Iglesias wrote:
> > Hello,
> > 
> > Since years I've been using a shell script of mine to shutdown my laptop
> > when battery is critical.  Convenient because I made it portable among
> > unix-like systems.  In the case of OpenBSD the script asks battery and
> > AC status to apm(4).
> > 
> > Now I gave a try to the apmd(8) -Z option but, so far, I couldn't make
> > it work in a reliable way.  I added to rc.conf.local:
> > 
> >  apmd_flags="-A -Z 20"
> > 
> > But, after doing some tests, sometimes it works, other it seems like
> > it's totally ignored.
> 
> Is your laptop plugged in during the tests? 

First of all, thank you for answering. :-)

The cable is plugged to a multiple socket that has and interrupter.  I
cut the energy from there while doing the tests, mainly because I'm
interested it work in that way.

I'm not an electricity expert but I think I understand why you're asking
that question.  If the AC cable is connected to the laptop, even when in
the other end it's not connected to the source, since the converter
holds some residual voltage could make the laptop think it's still
connected to the main source, right?  But when the source is cut (in the
way I explained above,) apm(4) correctly says the AC power is
"disconnected," that's why I assumed apmd(8) should not be tricked by
the converter.

> 
> Edgar
> 
> > 
> > Curious because power management seems to work fine in my T410.  It
> > sleeps, resumes and hibernates perfectly.  /var/log/messages and 'apmd
> > -d' don't show significant errors.  Do I need to set something else, add
> > some -t value to ampd command or some script to /etc/apm?
> > 
> > 
> > Walter
> > 

ampd(8) -Z option

[Walter Alejandro Iglesias]

Hello,

Since years I've been using a shell script of mine to shutdown my laptop
when battery is critical.  Convenient because I made it portable among
unix-like systems.  In the case of OpenBSD the script asks battery and
AC status to apm(4).

Now I gave a try to the apmd(8) -Z option but, so far, I couldn't make
it work in a reliable way.  I added to rc.conf.local:

 apmd_flags="-A -Z 20"

But, after doing some tests, sometimes it works, other it seems like
it's totally ignored.

Curious because power management seems to work fine in my T410.  It
sleeps, resumes and hibernates perfectly.  /var/log/messages and 'apmd
-d' don't show significant errors.  Do I need to set something else, add
some -t value to ampd command or some script to /etc/apm?


Walter

Re: bc(1) bug?

[Walter Alejandro Iglesias]

Hi Ingo,

On Sun, Jul 14, 2019 at 11:09:40PM +0200, Ingo Schwarze wrote:
> Hi Walter,
> 
> Walter Alejandro Iglesias wrote on Sun, Jul 14, 2019 at 09:57:11PM +0200:
> 
> > Lately I noticed that using bc(1) interactively I have to hit Enter
> > twice to get the result of an operation printed on screen.  Plus, a new
> > empty line is printed below the result.  Example:
> > 
> > $ bc
> > 2 + 1  (after hitting Enter twice the two lines below get printed)
> > 3
> > 
> > 2 + 2  (idem)
> > 4
> > 
> > quit
> > $
> > 
> > I can reproduce it in xterm and tty console.
> > 
> > (OpenBSD 6.5-current (GENERIC.MP) #125: Thu Jul 11 02:20:04 MDT 2019
> > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP)
> 
> To me, that smells a bit like it *might* be a variation of what
> got fixed with
> 
>   https://marc.info/?l=openbsd-cvs=156293980006370
> 
>   Date:   2019-07-12 13:56:28
> 
> Can you still reproduce with a kernel compiled after that commit?

You mean the problem could be in the kernel.  Do you think it'll help to
isolate the problem if I compile a newer kernel under the same userland
or upgrading to a newer snapshot is enough?

> 
> By the way, i cannot reproduce your issue.
> 
> Yours,
>   Ingo


Walter

bc(1) bug?

[Walter Alejandro Iglesias]

Hello everyone,

Lately I noticed that using bc(1) interactively I have to hit Enter
twice to get the result of an operation printed on screen.  Plus, a new
empty line is printed below the result.  Example:

$ bc
2 + 1  (after hitting Enter twice the two lines below get printed)
3

2 + 2  (idem)
4

quit
$

I can reproduce it in xterm and tty console.

(OpenBSD 6.5-current (GENERIC.MP) #125: Thu Jul 11 02:20:04 MDT 2019
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP)


Walter

About some smptd(8) log message

[Walter Alejandro Iglesias]

Hello Gilles,

When some spammer try to reach an invalid address in my server the log
says "Invalid recipient":

[...] smtp failed-command command="RCPT TO: " result="550 Invalid 
recipient: "

But, when the domain name part is valid (one of those included in my
"vdomains" and "valiases" tables), it appears a "Mailing list expansion
problem" message:

[...] smtp failed-command command="RCPT TO:" result="524 5.2.4 Mailing 
list expansion problem: "


In case what I assumed above is correct. :-)  Is the "Mailing list..."
message expected in this case?


Walter



# /etc/mail/smptd.conf

egress_int="em0"
server="server.roquesor.com"

table "aliases" file:/etc/mail/aliases
table "valiases"file:/etc/mail/valiases
table "vdomains"file:/etc/mail/vdomains
table "addresses"   file:/etc/mail/addresses
table "users"   file:/etc/mail/users

pki $server cert "/etc/ssl/server.crt"
pki $server key "/etc/ssl/private/server.key"

listen on lo0
listen on $egress_int port 25 tls pki $server
listen on $egress_int port 465 smtps pki $server auth \
senders  masquerade

action "local" mbox alias 
action "virtual" mbox virtual 
action "relay" relay

match from local for local action "local"
match from any for domain  action "virtual"
match from local mail-from  for any action "relay"
match auth from any mail-from  for any action "relay"

# End of file

Re: CVS: cvs.openbsd.org: src (maillog simplified)

[Walter Alejandro Iglesias]

Hello Gilles,

In article <20190101143249.ga41...@ams-1.poolp.org> Gilles Chehade 
 wrote:
> On Tue, Jan 01, 2019 at 01:14:54PM +0100, Walter Alejandro Iglesias wrote:
> > On Fri, Dec 21, 2018 at 06:59:58PM +0100, Gilles Chehade wrote:
> > > On Fri, Dec 21, 2018 at 06:56:57PM +0100, Walter Alejandro Iglesias wrote:
> > > > Hello Gilles,
> > > > 
> > > > In article <20181221145201.ga90...@ams-1.poolp.org> Gilles Chehade 
> > > >  wrote:
> > > > > On Fri, Dec 21, 2018 at 07:41:41AM -0700, Gilles Chehade wrote:
> > > > > > CVSROOT:  /cvs
> > > > > > Module name:  src
> > > > > > Changes by:   gil...@cvs.openbsd.org  2018/12/21 07:41:41
> > > > > > 
> > > > > > Modified files:
> > > > > >   usr.sbin/smtpd : smtp_session.c 
> > > > > > 
> > > > > > Log message:
> > > > > > start simplifying log lines, they're no longer intended to be 
> > > > > > parseable, we
> > > > > > have a reporting API for tools that want to analyze events, maillog 
> > > > > > is just
> > > > > > for us, hoomans.
> > > > > > 
> > > > > 
> > > > > that was not the best way to phrase my commit log ... sorry
> > > > > 
> > > > > i meant they're no longer intended to be friendlier to scripts than to
> > > > > humans: there will still be in a format that's easy to quickly script,
> > > > > but they will hold information easily readable by humans, not a lot of
> > > > > unrelated context infos so tools can generate dashboards out of single
> > > > > lines.
> > > > > 
> > > > > logs for humans, event reports for tools.
> > > > > 
> > > > 
> > > > Since long I've been greping IPs from spammers and attackers from
> > > > /var/log/maillog, /var/log/authlog and /var/log/daemon using a shell
> > > > script I wrote that automatically includes them in a file read by a pf
> > > > table.  In the case of maillog, it relies in the address="" and host=""
> > > > info currently included.
> > > > 
> > > > Will it appear sender's IP and hostname in /var/log/maillog after this
> > > > change?
> > > > 
> > > 
> > > yes, you'll still be able to grep that information from maillog
> > 
> > You selected carefully the words in your answer. :-)
> > 
> 
> not really, I don't know what your scripts do and how you wrote them.

I made this clear in my explanation below.  At least the relevant part.

> 
> the sender IP and hostname appear in the log, they are just not repeated
> on every single log line but that shouldn't prevent scripts from keeping
> track of them.

Also clear in my explanation that I understood this.

> 
> anyways, as stated in the commit log and my follow up message:
> 
> "we have a reporting API for tools that want to analyse events, maillog
>  is just for us, hoomans"
> 
> "logs for humans, event reports for tools"

System administrators (i.e. those who will use your software) are also
humans. :-)


> 
> the maillog format is going to go through many changes to simplify it,
> remove redundant information, add missing information, etc... basing a
> script on it is not recommended as we'll break them with every change.
> 
> > Indeed, I still can grep "IP" and "host" in maillog, but they are alone
> > in a first line and the only way to associate them with the following
> > lines containing the from= to= and result= (to know what "happened" with
> > that connection) is by using the connection id, what will *painfully*
> > overcomplicate my scripts.
> > 
> 
> As you imagine, I can't take into account individual scripts.
> 
> Other people have asked that the port or listener tag appear in lines.
> Should these appear on all lines too ?
> And the cipher ? and the authenticated user ?
> Why is the IP/host information more legitimate to be repeated than other
> information on every single line ?
> What about the fcrdns check which will appear on connect lines, does the
> check have to appear on every line now ?
> What about the spf check when it is added at some point ?
> 
> maillog is not a context-free format, where each individual line carries
> all of the information so you don't have to look at previous lines. Line
> should describe an event and carry informations related to THAT event.

Re: CVS: cvs.openbsd.org: src (maillog simplified)

[Walter Alejandro Iglesias]

On Fri, Dec 21, 2018 at 06:59:58PM +0100, Gilles Chehade wrote:
> On Fri, Dec 21, 2018 at 06:56:57PM +0100, Walter Alejandro Iglesias wrote:
> > Hello Gilles,
> > 
> > In article <20181221145201.ga90...@ams-1.poolp.org> Gilles Chehade 
> >  wrote:
> > > On Fri, Dec 21, 2018 at 07:41:41AM -0700, Gilles Chehade wrote:
> > > > CVSROOT:  /cvs
> > > > Module name:  src
> > > > Changes by:   gil...@cvs.openbsd.org  2018/12/21 07:41:41
> > > > 
> > > > Modified files:
> > > >   usr.sbin/smtpd : smtp_session.c 
> > > > 
> > > > Log message:
> > > > start simplifying log lines, they're no longer intended to be 
> > > > parseable, we
> > > > have a reporting API for tools that want to analyze events, maillog is 
> > > > just
> > > > for us, hoomans.
> > > > 
> > > 
> > > that was not the best way to phrase my commit log ... sorry
> > > 
> > > i meant they're no longer intended to be friendlier to scripts than to
> > > humans: there will still be in a format that's easy to quickly script,
> > > but they will hold information easily readable by humans, not a lot of
> > > unrelated context infos so tools can generate dashboards out of single
> > > lines.
> > > 
> > > logs for humans, event reports for tools.
> > > 
> > 
> > Since long I've been greping IPs from spammers and attackers from
> > /var/log/maillog, /var/log/authlog and /var/log/daemon using a shell
> > script I wrote that automatically includes them in a file read by a pf
> > table.  In the case of maillog, it relies in the address="" and host=""
> > info currently included.
> > 
> > Will it appear sender's IP and hostname in /var/log/maillog after this
> > change?
> > 
> 
> yes, you'll still be able to grep that information from maillog

You selected carefully the words in your answer. :-)

Indeed, I still can grep "IP" and "host" in maillog, but they are alone
in a first line and the only way to associate them with the following
lines containing the from= to= and result= (to know what "happened" with
that connection) is by using the connection id, what will *painfully*
overcomplicate my scripts.

I don't know what's the opinion of the rest about this change.  I'd
highly appreciate you to include again the IP on each line of info as
before. :-)

> 
> -- 
> Gilles Chehade   @poolpOrg
> 
> https://www.poolp.org tip me: https://paypal.me/poolpOrg


Walter

Re: CVS: cvs.openbsd.org: src

[Walter Alejandro Iglesias]

Hello Gilles,

In article <20181221145201.ga90...@ams-1.poolp.org> Gilles Chehade 
 wrote:
> On Fri, Dec 21, 2018 at 07:41:41AM -0700, Gilles Chehade wrote:
> > CVSROOT:  /cvs
> > Module name:  src
> > Changes by:   gil...@cvs.openbsd.org  2018/12/21 07:41:41
> > 
> > Modified files:
> >   usr.sbin/smtpd : smtp_session.c 
> > 
> > Log message:
> > start simplifying log lines, they're no longer intended to be parseable, we
> > have a reporting API for tools that want to analyze events, maillog is just
> > for us, hoomans.
> > 
> 
> that was not the best way to phrase my commit log ... sorry
> 
> i meant they're no longer intended to be friendlier to scripts than to
> humans: there will still be in a format that's easy to quickly script,
> but they will hold information easily readable by humans, not a lot of
> unrelated context infos so tools can generate dashboards out of single
> lines.
> 
> logs for humans, event reports for tools.
> 

Since long I've been greping IPs from spammers and attackers from
/var/log/maillog, /var/log/authlog and /var/log/daemon using a shell
script I wrote that automatically includes them in a file read by a pf
table.  In the case of maillog, it relies in the address="" and host=""
info currently included.

Will it appear sender's IP and hostname in /var/log/maillog after this
change?


Walter

Re: what would a POP3s daemon best look like?

[Walter Alejandro Iglesias]

On Sun, Nov 04, 2018 at 05:38:42AM -0700, Todd C. Miller wrote:
> On Sun, 04 Nov 2018 12:26:27 +0100, Walter Alejandro Iglesias wrote:
> 
> > I've been assuming that running pop3d(8) from ports, listening in 995
> > only and with 110 port firewalled my passwords aren't traveling in plain
> > text.  Am I assuming right?
> 
> Port 995 is pop3 protocol over TLS/SSL so that should be safe enough.

Then, as an idea for Peter,

Time ago I sent a patch to Sunil Nimmagadda to allow pop3d read an
optional certs location, he corrected and committed the patch.  In that
opportunity he mentioned me that he wasn't hacking pop3d anymore since
he himself stopped using it because he considered it severely limited.
Personally I like simplicity, I still use pop3d(8) but I'm not a
developer, I'm not skilled enough to hack it and maintain it.

If Peter is willing to, perhaps pop3d(8) could be a good start point.

If allowing pop connections by default through 110 port is not desirable
perhaps it would be fine to implement an only TLS pop3 daemon
(deliberately refusing non TLS connections over 110).  In case this is
possible, that would be a fine simple and secure pop3 daemon for OpenBSD
base.

> 
>  - todd

Walter

Re: what would a POP3s daemon best look like?

[Walter Alejandro Iglesias]

Hi Todd,

Not an expert here and just to be sure, :-)

In article <21bf906b4c6c6...@sudo.ws> Todd C. Miller  
wrote:
> I don't think there is much interest in having a pop3 daemon in
> base due to the use of plain-text passwords

I've been assuming that running pop3d(8) from ports, listening in 995
only and with 110 port firewalled my passwords aren't traveling in plain
text.  Am I assuming right?


Walter

Re: kernel panic while reproducing video with mpv

[Walter Alejandro Iglesias]

Hi Visa,

On Sun, Jun 24, 2018 at 05:54:15PM +, Visa Hankala wrote:
> On Sun, Jun 24, 2018 at 12:37:45PM +0200, Walter Alejandro Iglesias wrote:
> > panic: mtx 0x81c86470: locking against myself
> > Stopped at  db_enter+0x12:  popq%r11
> > TIDPIDUID PRFLAGS PFLAGS  CPU  COMMAND
> >  104021  96401   1000 0x3  0x4002  mpv
> > *402610  50624   10000x32  00K Xorg
> >   
> > db_enter() at db_enter+0x12
> > panic() at panic+0x138
> > __mtx_enter_try(53b9235709d40154) at __mtx_enter_try+0xb5
> > _mtx_enter(81cf3e60,81a5d6a2,0) at _mtx_enter+0x5a
> > printf(c9ef1007dec621e0) at printf+0x70
> > witness_checkorder(2e4447d1b3cbb9af,81c2ac7c,32a,0,81da6d00)
> >  at 
> > witness_checkorder+0x943
> > ___mp_lock(8000330cd760,d,7) at ___mp_lock+0x70
> > selwakeup(e80faaebded7c1a2) at selwakeup+0x9c
> > ptsstart(8ce5939828d5e23) at ptsstart+0x79
> > tputchar(174549bf676e909c,80afa400) at tputchar+0x85
> > kputchar(75d50501b895e9e4,0,81a5d6a2) at kputchar+0x91
> > kprintf() at kprintf+0xe8
> > printf(c9ef1007dec621e0) at printf+0x85
> > witness_checkorder(2e4447d1b3cba2fe,81af9df1,298,81c8a678,ff
> > ff81c8a688) at witness_checkorder+0x943
> > end trace frame: 0x80003302e978, count: 0
> 
> If the panic happens again, please run the following commands in ddb(4)
> and post the output:
> 
> show locks
> show all locks

The true is it happend twice.  On the first one fsck(8) couldn't recover
my root file system.  After rebooting I couldn't even log in (as user or
root) and I had to reinstall.  That's way I'm not confident about
"voluntary" reproducing the bug. :-)  But if it happens again take for
sure I'll send you the output of those commands (and per cpu traces).

> 
> It is not clear from the stack trace why the system begins to report
> a lock order problem in the first place (the first witness_checkorder
> and the printf at the end of the stack trace).
> 
> The panic itself is related to the problem of using other kernel
> subsystems from WITNESS. I will try to make a fix that should prevent
> the panic in most cases.


Thanks!

Walter

kernel panic while reproducing video with mpv

[Walter Alejandro Iglesias]

Hello,

I had a kernel panic while reproducing a video with mpv.

It's my first kernel panic with OpenBSD, so I didn't know how to use
ddb(4).  Since I'm running my http and smtp server in this machine I
cannot entertain myself too much reproducing the panic to get more info.
That's why I don't include the per cpu trace and other additonal info as
explained in ddb.html, sorry!  But, if you need it let me knonw and I'll
try my best.


Message automatically dumped:
===
panic: mtx 0x81c86470: locking against myself
Stopped at  db_enter+0x12:  popq%r11
TIDPIDUID PRFLAGS PFLAGS  CPU  COMMAND  
 
 104021  96401   1000 0x3  0x4002  mpv  
  
*402610  50624   10000x32  00K Xorg 
  
db_enter() at db_enter+0x12
panic() at panic+0x138
__mtx_enter_try(53b9235709d40154) at __mtx_enter_try+0xb5
_mtx_enter(81cf3e60,81a5d6a2,0) at _mtx_enter+0x5a
printf(c9ef1007dec621e0) at printf+0x70
witness_checkorder(2e4447d1b3cbb9af,81c2ac7c,32a,0,81da6d00) at 
witness_checkorder+0x943
___mp_lock(8000330cd760,d,7) at ___mp_lock+0x70
selwakeup(e80faaebded7c1a2) at selwakeup+0x9c
ptsstart(8ce5939828d5e23) at ptsstart+0x79
tputchar(174549bf676e909c,80afa400) at tputchar+0x85
kputchar(75d50501b895e9e4,0,81a5d6a2) at kputchar+0x91
kprintf() at kprintf+0xe8
printf(c9ef1007dec621e0) at printf+0x85
witness_checkorder(2e4447d1b3cba2fe,81af9df1,298,81c8a678,ff
ff81c8a688) at witness_checkorder+0x943
end trace frame: 0x80003302e978, count: 0


dmesg:
===
OpenBSD 6.3-current (GENERIC.MP) #48: Fri Jun 22 14:11:27 MDT 2018
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 6210174976 (5922MB)
avail mem = 5960577024 (5684MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xe0010 (78 entries)
bios0: vendor LENOVO version "6IET85WW (1.45 )" date 02/14/2013
bios0: LENOVO 2537EY8
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT APIC MCFG HPET ASF! SLIC BOOT SSDT TCPA SSDT S
SDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) IGBE(S4) EXP1(S4) EXP2(S4) EXP3(S4) EXP4
(S4) EXP5(S4) EHC1(S3) EHC2(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz, 2793.56 MHz
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CF
LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,
PERF,ITSC,SENSOR,ARAT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 132MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz, 2793.00 MHz
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CF
LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,
PERF,ITSC,SENSOR,ARAT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz, 2793.00 MHz
cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CF
LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,
PERF,ITSC,SENSOR,ARAT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 5 (application processor)
cpu3: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz, 2793.00 MHz
cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CF
LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,
PERF,ITSC,SENSOR,ARAT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 2, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins
, remapped to apid 1
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (PEG_)
acpiprt2 at acpi0: bus 2 (EXP1)
acpiprt3 at acpi0: bus 3 (EXP2)
acpiprt4 at acpi0: bus -1 (EXP3)
acpiprt5 at acpi0: bus 5 (EXP4)
acpiprt6 at acpi0: bus 13 (EXP5)
acpicpu0 at acpi0: C3(350@245 mwait.3@0x20), C2(500@205 mwait.3@0x10), C1(1000@3
 mwait.1), PSS
acpicpu1 at 

I got smtpd.conf working thanks to the man page

[Walter Alejandro Iglesias]

Just in case it could be useful to others.

After upgrading the snaptshot requiring the new version of smtpd.conf
it happend that the new rules I'd written (included the last one Gilles
passed me) were all wrong.

I could get it working thanks to the man page.  The result:

# OLD
accept from local for local alias  deliver to mbox
accept from any for domain  virtual  deliver to mbox
accept from local sender  for any relay


# FIST ATTEMPT (smtpd -n told me the three last lines were wrong)
action local_users mbox alias 
action remote_users relay

match from local for local apply local_users
match from any for domain  virtual  apply local_users
match from local sender  for any apply remote_users
match auth from any sender  for any apply remote_users


# NOW WORKING
action "local" mbox alias 
action "virtual" mbox virtual 
action "relay" relay

match from local for local action "local"
match from any for domain  action "virtual"
match mail-from  for any action "relay"
match auth mail-from  for any action "relay"


My advice to others is not to pay attention to anything but the man
page, checking one by one each option you used in the old configuration,
if it still exists, if it was replaced and finally *where* to pass it,
if to match or to action.  Doing it in that order you'll probably go
faster. :-)

As you see above I had to replace "sender" for "mail-from" and to create
a third action to pass the virtual aliases table that in the first
attempt I'd wrongly included it in the match.

Re: Checking my new smtpd.conf syntax

[Walter Alejandro Iglesias]

On Sat, May 26, 2018 at 12:35:57PM +0200, Walter Alejandro Iglesias wrote:
> On Sat, May 26, 2018 at 08:15:18AM +0200, Gilles Chehade wrote:
> > > Gilles, I also saw the "ca" directive.  I've been using the acme
> > > certificates in pki directives, can I use them in the "ca" directive
> > > too? (any advantage in doing this?)
> > > 
> > 
> > don't touch a knob if you don't KNOW that you absolutely need it.
> > 
> > I know why some people would like to use a custom CA certificate instead
> > of the one shipped with the system, I don't know why YOU should do it so
> > if you are asking I can only guess you are going to break your setup.
> 
> First of all, each one is responsible of what they do with their system,
> it's the nature of free software, isn't it?  Don't be afraid, if I break
> my setup I won't sue you. :-)
> 
> In the past I used the defunct StartSSL(TM) certificates with Apache and
> Sendmail during years.  In the case of a mail server I thought that, by
> logic, to present something that certificates your identity (what a CA
> is for, isn't it?) should be one among the more acceptable ways to avoid
> your messages be considered SPAM.
> 
> What I'm not clear about is what Let's Encrypt does (differently).  And,
> logically, I'm not clear about what your software does in this case.
> And over all I'm not clear about (and probably nobody is at this stage)
> what mail servers do and why with their SPAM filters.  That was the aim
> of my question.
> 
> By the way, your messages got to my server but not to misc@ (at least I
> can't not read them through gmane), I guess they got trapped in spamd
> daemon.

Let me add something more about what I know.

Each software (i.e. apache, ngnix, uw-imap, sendmail, etc) requires a
different setup to get the certificates working.  In some cases you need
to put chain and cert in one file, in others (uw-imap) you need to
include the key in a same one file.

I just expected you could tell me (or point me where this is documented)
what to do in opensmptd case.  The explanaintion in starttls(8) isn't
enough.

For example, what does the smptd.conf "ca" directive expect?, a root
certificates bundle?  Intermediate certificates?  What does the software
use in case you don't set this option?, the system provided
/etc/ssl/cert.pem?

I'll tell you what I been doing so far.  When time ago I started using
opensmtpd with the certs downloaded with acme-client, *after some trial
and error* I got it working with this set up:

Here I use the "full chain" certificate:

  pki $server cert "/etc/ssl/server.crt"

Here the key:

  pki $server key "/etc/ssl/private/server.key"

Re: Checking my new smtpd.conf syntax

[Walter Alejandro Iglesias]

On Sat, May 26, 2018 at 08:15:18AM +0200, Gilles Chehade wrote:
> > Gilles, I also saw the "ca" directive.  I've been using the acme
> > certificates in pki directives, can I use them in the "ca" directive
> > too? (any advantage in doing this?)
> > 
> 
> don't touch a knob if you don't KNOW that you absolutely need it.
> 
> I know why some people would like to use a custom CA certificate instead
> of the one shipped with the system, I don't know why YOU should do it so
> if you are asking I can only guess you are going to break your setup.

First of all, each one is responsible of what they do with their system,
it's the nature of free software, isn't it?  Don't be afraid, if I break
my setup I won't sue you. :-)

In the past I used the defunct StartSSL(TM) certificates with Apache and
Sendmail during years.  In the case of a mail server I thought that, by
logic, to present something that certificates your identity (what a CA
is for, isn't it?) should be one among the more acceptable ways to avoid
your messages be considered SPAM.

What I'm not clear about is what Let's Encrypt does (differently).  And,
logically, I'm not clear about what your software does in this case.
And over all I'm not clear about (and probably nobody is at this stage)
what mail servers do and why with their SPAM filters.  That was the aim
of my question.

By the way, your messages got to my server but not to misc@ (at least I
can't not read them through gmane), I guess they got trapped in spamd
daemon.


> 
> 
> -- 
> Gilles Chehade
> 
> https://www.poolp.org  @poolpOrg


Walter

Re: Checking my new smtpd.conf syntax

[Walter Alejandro Iglesias]

On Fri, May 25, 2018 at 03:58:59PM +0300, Consus wrote:
> On 14:31 Fri 25 May, Gilles Chehade wrote:
> > On Fri, May 25, 2018 at 02:20:50PM +0200, Walter Alejandro Iglesias wrote:
> > > Could someone tell me if my changes below are OK. :-)
> > > 
> > > The part I'm not clear is I read in current.html remote authenticated
> > > users need a explicit rule.  Do I need to add some "match auth" rule?
> > > 
> > 
> > yes.
> > 
> > before, "from local" would match authenticated users as if they had sent
> > mail from the local machine but this led to being unable to express some
> > setups where depending on the source you want to relay to different hubs
> > even though users are authenticated.
> > 
> > 
> > With this:
> > 
> > > match from local for local apply local_users
> > > match from any for domain  virtual  apply local_users
> > > match from local sender  for any apply remote_users
> > 
> > you need an additonal rule such as:
> > 
> > match auth from any sender  for any apply remote_users
> > 
> > 
> > because:
> > 
> > > #accept from local sender  for any relay
> > 
> > no longer matches authenticated users
> 
> Ain't it "action local_users" instead of "apply local_users"? The man
> page states "action".

I took the "apply" from here:

  https://undeadly.org/cgi?action=article;sid=20180430122930

Now reading this:

  https://poolp.org/posts/2018-05-21/switching-to-opensmtpd-new-config/

I see I also have to change the "certificate" keyword to "cert" here:

  pki $server cert "/etc/ssl/server.crt"


Gilles, I also saw the "ca" directive.  I've been using the acme
certificates in pki directives, can I use them in the "ca" directive
too? (any advantage in doing this?)



Walter

Checking my new smtpd.conf syntax

[Walter Alejandro Iglesias]

Could someone tell me if my changes below are OK. :-)

The part I'm not clear is I read in current.html remote authenticated
users need a explicit rule.  Do I need to add some "match auth" rule?


# /etc/mail/smptd.conf

egress_int="em0"
server="server.roquesor.com"

table aliases   file:/etc/mail/aliases
table valiases  file:/etc/mail/valiases
table vdomains  file:/etc/mail/vdomains
table addresses file:/etc/mail/addresses
table users file:/etc/mail/users

pki $server certificate "/etc/ssl/server.crt"
pki $server key "/etc/ssl/private/server.key"

listen on lo0
listen on $egress_int port 25 tls pki $server
listen on $egress_int port 465 smtps pki $server auth \
senders  masquerade

# Old
#accept from local for local alias  deliver to mbox
#accept from any for domain  virtual  deliver to mbox
#accept from local sender  for any relay

# New
action local_users mbox alias 
action remote_users relay

match from local for local apply local_users
match from any for domain  virtual  apply local_users
match from local sender  for any apply remote_users

# End of file

Re: Viewport for man.openbsd.org -- readability on phones

[Walter Alejandro Iglesias]

In article <20180518004729.gl68...@athene.usta.de> Ingo Schwarze 
 wrote:
> Hi Aner,
> 
> Aner Perez wrote on Thu, May 17, 2018 at 06:32:44PM -0400:
> > On 05/17/2018 05:22 PM, x...@dr.com wrote:
> >> "Ingo Schwarze"  wrote:
> 
> >>> Absolutely not.
> >>> Mandoc output is not optimized for any device.
> >>>
> >>> Which elements or rules in the current HTML or CSS code
> >>> make you think it is optimized or it discriminates against
> >>> any device?
> 
> >> I don't know which element or rule is the problem, however
> >> if I delete mandoc.css the text does fill the screen.
> >> 
> >> I understand that what I am trying to do is not supported,
> >> so I'll do something else instead.
> 
> > First non-comment line of mandoc.css says:
> > 
> > html {max-width: 100ex; }
> > 
> > Removing this line allows the use of the full browser width.
> 
> That is a very useful bit of information.
> Thanks for investigating and reporting it.
> 
> For testing purposes, i removed that line from
>   https://man.openbsd.org/mandoc.css
> 
> xcv@, could you check with your phone whether this solves
> your original issue?
> 
> > I'm sure that it was put there for a reason
> > (maybe to approximate the width of a terminal?).
> 
> Correct.  The original reason was that for -T ascii and -T utf8
> output, the default is -O width=78.  The reason for that is that
> it's conventional wisom in typography that readability of text
> suffers with excessive column width - even though some recent
> research raises doubts whether that is really true.  Either way,
> people tend to feel strongly about it.

If text is too wide, each time your sight jumps from the end to the
beginning of the other line it loses track of in which one it was.  When
it's too narrow (as used in news papers) your sight has to jump
continuously.  That's why in books you generally see lines not narrower
than 60 columns and not wider than 78, that's the comfortable range.

Perhaps I'm wrong assuming this happens to other people.  I'd like to
know if that recent research you mention took in care nowadays most
people read no more than one line at a time. :-)  Web sites are designed
to look pretty, text is there just for SEO.  I mean the oppinion of most
people about what is comfortable while reading doesn't tell the truth.

> 
> I must say i never particularly liked that line in the CSS file.
> It always felt like fiddling with details that it might be better
> not to touch, given that display devices running browsers differ
> more than terminal emulators.  And here we are with a suspicion
> that it actually causes accessibility issues, even if the suspicion
> is still unconfirmed...

It's not a mandoc problem.  That line is a workaround, so even when I
prefer that behavior I'm not against removing it.

> 
> Depending on the feedback i get here with respect to how
>   https://man.openbsd.org/
> now looks, i shall consider deleting the offending line for good.
> 
> In general, i like the idea of making things better by *removing*
> harmful tweaks rather than adding new goo...

Have you added apple-touch-icon.png in all required sizes?  No?  Why do
you resist to innovation, to "new technologies"?  Here you have a guide:

https://developer.apple.com/library/content/documentation/AppleApplications/Reference/SafariWebContent/ConfiguringWebApplications/ConfiguringWebApplications.html

:-)


> 
> Yours,
>   Ingo
> 
> 


Walter

Re: Do I need slaacd(8) up and running?

[Walter Alejandro Iglesias]

On Sat, Nov 11, 2017 at 04:57:14PM -0700, Theo de Raadt wrote:
> >On Sat, Nov 11, 2017 at 05:58:59AM -0700, Theo de Raadt wrote:
> >> >A question to the experts here.
> >> >
> >> >My home router (a crappy one provided by my ISP) has ipv6 disabled, at
> >> >least it's what its guied configuration tells me. :-)  And I have ipv6
> >> >disabled in all my LAN machines.  The laptop I use with OpenBSD has
> >> >slaacd(8) up and running by default, even when I didn't configure any
> >> >interface to use ipv6 at install time.
> >> >
> >> >Under the above conditions, do I still need slaacd running?
> >> 
> >> Yes, absolutely.
> >> 
> >> Otherwise one day you will configure up v6 on an interface and
> >> come whining about how your custom configuration isn't do inet6
> >> boohoohoo.
> >
> >OK.  You assume I'm an asshole.
> >
> >> 
> >> You need it.  And don't go writing some balony blog saying you don't
> >> need it.
> >
> >I don't need blogs. :-)
> >
> >
> >Look, I'm very happy with OpenBSD (*honestly*) in the technical as well
> >as in the human aspect.  The *only one* negative point I found till now
> >in this project is your attitude.  The next time you want to insult me
> >do it in private, in that way you won't harm the project (taking in care
> >the other people working hard on it).
> 
> Terribly sad you are such a sensitive soul.

Uh, your sarcasms hurt my delicate soul. :-)

I don't usually come here to whine.  I've always kept my systems as
default as possible.  I've never written any article about OpenBSD.
Obviously it's not about me and *that's the bad news*.  Whether or not
you're right about users in general, there are more than one OS out
there with long tradition and experience in developing with the
assumption users are a bunch of irresponsible idiots.  And they count
with a stronger infrastructure than yours.  It's not clever to compete
with those monsters using their same strategy.

Re: Do I need slaacd(8) up and running?

[Walter Alejandro Iglesias]

On Sat, Nov 11, 2017 at 05:58:59AM -0700, Theo de Raadt wrote:
> >A question to the experts here.
> >
> >My home router (a crappy one provided by my ISP) has ipv6 disabled, at
> >least it's what its guied configuration tells me. :-)  And I have ipv6
> >disabled in all my LAN machines.  The laptop I use with OpenBSD has
> >slaacd(8) up and running by default, even when I didn't configure any
> >interface to use ipv6 at install time.
> >
> >Under the above conditions, do I still need slaacd running?
> 
> Yes, absolutely.
> 
> Otherwise one day you will configure up v6 on an interface and
> come whining about how your custom configuration isn't do inet6
> boohoohoo.

OK.  You assume I'm an asshole.

> 
> You need it.  And don't go writing some balony blog saying you don't
> need it.

I don't need blogs. :-)


Look, I'm very happy with OpenBSD (*honestly*) in the technical as well
as in the human aspect.  The *only one* negative point I found till now
in this project is your attitude.  The next time you want to insult me
do it in private, in that way you won't harm the project (taking in care
the other people working hard on it).

Do I need slaacd(8) up and running?

[Walter Alejandro Iglesias]

A question to the experts here.

My home router (a crappy one provided by my ISP) has ipv6 disabled, at
least it's what its guied configuration tells me. :-)  And I have ipv6
disabled in all my LAN machines.  The laptop I use with OpenBSD has
slaacd(8) up and running by default, even when I didn't configure any
interface to use ipv6 at install time.

Under the above conditions, do I still need slaacd running?

Re: mandoc output paper size

[Walter Alejandro Iglesias]

In article  Mike Williams 
 wrote:
> Hiya
> 
> On 10/27/17 14:31, Ingo Schwarze wrote:
> > [ sending this particular one back to the list
> >   because it contains something useful for everyone and nothing private ]
> 
> Replying to list to archive comments even if not acted on.
> 
> > Hi Jan,
> >
> > Jan Stary wrote on Fri, Oct 27, 2017 at 12:46:00PM +0200:
> >
> >> I produced a PS output with "man -Tps rm > rm.ps",
> >> with output paper set to a3, a4, and a5 in man.conf.
> >> This results, respectively, in
> >>
> >>  %%DocumentMedia: Default 841 1190 0 () ()
> >>  %%DocumentMedia: Default 595 841 0 () ()
> >>  %%DocumentMedia: Default 419 595 0 () ()
> >>
> >> which apparently are the right dimensions. However,
> >> the Minolta will print all of them on A4 paper,
> >> although it does have a stash of A3 and A5 too.
> >>
> >> That's where I thought it might take a hint from the DSC comment,
> >> if I changed the "Default" to "A3" or "A4" or "A5", or if mandoc(1)
> >> itself put that in the DSC comments. I rewrote it manually before
> >> each printing, but the Minolta still prints them all on an A4:
> >
> > That's interesting, but anecdotal.  It is neither surprising that
> > a specific printer selects paper as configured (in whichever way),
> > as opposed to inspecting fikes it is sent; nor would it be surprising
> > if other printers, or even the same one, or printer drivers on the
> > print server, could be configured to inspect the contents of
> > PostScript files to select paper.
> >
> > The trouble is, i just don't know what firmwares and softwares do,
> > what they should do according to standards, and where to look for
> > standards in this respect.
> >
> > Does anybody else know?
> 
> The DSC comments are not part of the PS specification (the reference 
> manual quoted earlier) and a PS interpreter would not normally take 
> notice of the them.  The comments are aimed at document printing systems 
> which can just look for the comments and manage the printing of the file 
> amongst other files and a range of available printers - such as queuing 
> for a printer which supports/has the media listed by the DSC comment.
> 
> Some PS interpreters may look for and act on the comments, but this 
> would not be "standard" behaviour.  For the %%DocumentMedia: comment the 
> name used is for human consumption, it is the numeric values that are 
> used for any media handling decisions.  The %%DocumentMedia: comment 
> lists all media sizes used by the document but does not say which page 
> uses which size of media so could not be used to select media for any 
> particular page.  If the  media size is important for a page then there 
> should be a PS setpagedevice call like the following:
> 
> <>setpagedevice

In my other message I was about to mention that in the document
generated by groff I inspected, besides the comment, I found this other
line:

  %%BeginFeature: *PageSize Default
  << /PageSize [ 595 842 ] /ImagingBBox null >> setpagedevice

It's in the place I put an image.  I tried modifying the values here too
to see if this line was taken in care by gv too, but it seems it's not.


> 
> The PS interpreter will perform media selection based on the values - 
> use matching media, next largest, scale/rotate content, ask operator, 
> etc. - see section 6.2.1 of the PS reference manual for way more detail.
> 
> This is what is needed for the Minolta printer to use the other media 
> sizes it has available.  A quick edit of the PS file to add the above 
> line with the appropriate media sizes for A3 or A5 should prove that.
> 
> Basically don't rely on DSC comments to do media selection.
> 
> I don't know gv but it will be working as a virtual printer with some 
> standard media sizes to use when rendering a file.  It sounds like it 
> adds the list of media from any %%DocumentMedia: comments to provide 
> additional media sizes it may not have by default.  Ah, in the State 
> menu there is the option "Respect document structure".  If I unselect 
> this the page size used by gv changes from the letter used in the -Tps 
> output to the default A4 used by gv.
> 
> It may be useful to use a media name such as man-A4, man-letter, etc. 
> (to indicate the source of the file and media size used) for any systems 
> that do process the DCS comments.  That will avoid duplicates appearing 
> in generated media lists.
> 
> As for PDF, no there is no way to name the media size being used.  There 
> is no equivalent of the DSC comments for PDF.  Media selection is always 
> done based the dimensions in the /MediaBox array, the same way as the 
> /PageSize array in PS, and it is up to the processor to decide how to 
> handle the media size request.
> 
> Finally, the -Tpdf output is not a valid PDF.  It is missing the endobj 
> keyword from several of the object definitions.  This will cause 
> warnings or errors when processing.  I 

Re: mandoc output paper size

[Walter Alejandro Iglesias]

In article <20171027104221.gd9...@www.stare.cz> Jan Stary  wrote:
> On Oct 27 12:12:21, w...@roquesor.com wrote:
> > In article <20171026193138.ga41...@www.stare.cz> Jan Stary  
> > wrote:
> > > > > > In the ps file generated by mandoc you should have this line:
> > > > > > 
> > > > > >   %%DocumentMedia: Default 595 841 0 () ()
> > > > > > 
> > > > > > Where 595 841 correspond to A4.  If you set output paper to "letter"
> > > > > > that line will say:
> > > > > > 
> > > > > >   %%DocumentMedia: Default 612 790 0 () ()
> > > 
> > > Yes. It seems that these are just _comments_ to the PS interpreter
> > > and the "Default" is just an arbitrary given name, right?
> > > (Sorry, I don't know the language.) So GV just shows that,
> > > but it does not _determine_ the actual media size, right?
> > > Looking at term_ps.c, mandoc writes "Default ... " for every paper size.
> > > 
> > 
> > First of all, I'm just a user like you trying to figure out how things
> > work.  So, don't expect from me some deep analysis, for that Ingo is the
> > right person.
> > 
> > I answered you - based in what I intuitively observed - that mandoc
> > honors the paper size, and explained you why I think so.
> > 
> > I know about postcript language as much as you, as well as what gv takes
> > in care to print the document on the screen, so first I grep in the
> > ps file for 'a4|letter' strings and got nothing, then searching on the
> > Internet I found the dots equivalence and repeated the search this time
> > using '595 841|612 790'.  I did the same with documents generated by GNU
> > roff.  I found the "comment" I mentioned in the other message, so
> > I opened the ps file with vi(1), changed those numbers, and then
> > I opened the modified file with gv.  That's how I found out gv takes in
> > care that "comment" to figure out physical page dimensions.
> 
> Apparently, it does not: the dimensions are given explicitly in e.g.
> "%%DocumentMedia: Default 595 841 0 () ()", and the "Default"
> could just as well be "Foobar", as Ingo explained.
> 

That's the "comment" we're talking about since the beginning of the
thread, aren't we?  As I told you what I modified to do the test was the
numbers.

> > Finally, "default" means "default". :-)  Perhaps (guessing again), since
> > page size use is related to region settings, who designed postscript
> > (hence gv) thought convenient to honor some wide system setting (based
> > on locale?).
> 
> With output paper set to A3, A4, A5 in man.conf, "man -Tps rm > rm.ps"
> will produce a PostScript file with the correct dimensions,
> calling all the formats "Default". A printer (such us my Minolta)
> will print them all on A4, although it does have A3 and A5 paper too.
> Changing the "%%DocumentMedia: Default ..." line manualy to "A3" or "A5"
> does not change that.
> 
> I am not saying mandoc should write A3 or A4 or A5 instead of Default
> (it's the actual dimensions that matter), but perhaps such a DSC comment
> might help some appications. Apparently not GV, which just repeats the name,
> and not my Minolta, which prints on A4 anyway.

You know, too much people developing software without caring about what
others did before.  Who developed your Minolta software is not an
exception. ;-)


> 
> Jan
> 
> 

Walter

Re: mandoc output paper size

[Walter Alejandro Iglesias]

In article <20171026193138.ga41...@www.stare.cz> Jan Stary  
wrote:
> > > > In the ps file generated by mandoc you should have this line:
> > > > 
> > > >   %%DocumentMedia: Default 595 841 0 () ()
> > > > 
> > > > Where 595 841 correspond to A4.  If you set output paper to "letter"
> > > > that line will say:
> > > > 
> > > >   %%DocumentMedia: Default 612 790 0 () ()
> 
> Yes. It seems that these are just _comments_ to the PS interpreter
> and the "Default" is just an arbitrary given name, right?
> (Sorry, I don't know the language.) So GV just shows that,
> but it does not _determine_ the actual media size, right?
> Looking at term_ps.c, mandoc writes "Default ... " for every paper size.
> 

First of all, I'm just a user like you trying to figure out how things
work.  So, don't expect from me some deep analysis, for that Ingo is the
right person.

I answered you - based in what I intuitively observed - that mandoc
honors the paper size, and explained you why I think so.

I know about postcript language as much as you, as well as what gv takes
in care to print the document on the screen, so first I grep in the
ps file for 'a4|letter' strings and got nothing, then searching on the
Internet I found the dots equivalence and repeated the search this time
using '595 841|612 790'.  I did the same with documents generated by GNU
roff.  I found the "comment" I mentioned in the other message, so
I opened the ps file with vi(1), changed those numbers, and then
I opened the modified file with gv.  That's how I found out gv takes in
care that "comment" to figure out physical page dimensions.

As far as I understand postscript draws page contents using coordinates
and using the postscript dot as unit (as Ingo explained).  What gv does
is just trying to figure out the best way to print the document on
screen; when you select A4|Letter in the menu it only modifies the page,
the rest of dimensions stay the same.  Ingo will correct me if I'm wrong
about this, we're talking specifically about how gv shows you the
document in screen, it shouldn't affect how the document is printed on
paper (what I *guess* gv does in this case is to send the postscript
file "as is" to lpr or cups.)

Finally, "default" means "default". :-)  Perhaps (guessing again), since
page size use is related to region settings, who designed postscript
(hence gv) thought convenient to honor some wide system setting (based
on locale?).


> Jan
> 
> 

Walter

Sent here by mistake (instead to bugs@) Sorry!

[Walter Alejandro Iglesias]

In article <a67500574d104...@server.roquesor.com> Walter Alejandro Iglesias 
<w...@roquesor.com> wrote:
> Hi Ruben,
> 
> In article 
> <caenp9cg+b-5b+8r3w9eaebodaxeybrdhg7jhfgq2ascrbfg...@mail.gmail.com> Ruben 
> Miller <rubenmil...@gmail.com> wrote:
> > In article 
> > <CAEnp9CEpPEJxkWkxLu1qmP8qTA4Ti4+6hCFrGqYy1+WZ0dBy=a...@gmail.com>
> > Ruben Miller <rubenmil...@gmail.com> wrote:
> > >The speed is not a problem, since the bug is triggered because cwm raise
> > > two windows in every cycle.
> > > Just start the cycle with seamonkey selected, so it's always the previous
> > > window.
> > 
> > Just in case, the idea is cycling without releasing ALT, so the client with
> > WM_TAKE_FOCUS is always behind the new one.
> 
> First of all, I'm not a developer but since I made that diff I'm trying
> to help.
> 
> No idea in which way it's related but I could easily reproduce the issue
> you describe after setting back SNA acceleration in my xorg.conf (since
> my graphic card has some issue with the default acceleration I have to
> use UXA.)
> 
> Wait to Okan Demirmen (cwm maintainer) to get a good answer. :-)
> 
> 

I sent this here by mistake.


Sorry!

Re: cwm 6.2: Windows losing focus while cycling (ALT-TAB)

[Walter Alejandro Iglesias]

Hi Ruben,

In article  
Ruben Miller  wrote:
> In article 
> Ruben Miller  wrote:
> >The speed is not a problem, since the bug is triggered because cwm raise
> > two windows in every cycle.
> > Just start the cycle with seamonkey selected, so it's always the previous
> > window.
> 
> Just in case, the idea is cycling without releasing ALT, so the client with
> WM_TAKE_FOCUS is always behind the new one.

First of all, I'm not a developer but since I made that diff I'm trying
to help.

No idea in which way it's related but I could easily reproduce the issue
you describe after setting back SNA acceleration in my xorg.conf (since
my graphic card has some issue with the default acceleration I have to
use UXA.)

Wait to Okan Demirmen (cwm maintainer) to get a good answer. :-)

Re: mandoc output paper size

[Walter Alejandro Iglesias]

On Thu, Oct 26, 2017 at 07:24:43PM +0200, Ingo Schwarze wrote:
> Hi Walter,
> 
> Walter Alejandro Iglesias wrote on Thu, Oct 26, 2017 at 05:44:16PM +0200:
> 
> > I have files generated with GNU roff that defaults to letter size.
> 
> That's the upstream (GNU troff) default when you compile GNU troff
> from the git repository with automake and autoconf.  If i understand
> correctly, it is the GNU troff default because it is also the default
> used by GNU autoconf in general.
> 
> > This doesn't happen on Linux, I ignore why.
> 
> I doubt this has anything to do with Linux (neither the kernel nor
> whatever C library or userland applications are used).  But it may
> depend on whatever operating system distribution you are using.  It
> is well-known that many Lnux distributions engage in tweaking
> upstream defaults, even those settings that are more or less a
> matter of personal preference.

By "linux" I meant distributions.

> 
> > This is set in DESC config files.
> > 
> > $ grep -ER 'papersize (letter|a4)' /usr/local/share/groff/*
> > /usr/local/share/groff/1.22.3/font/devdvi/DESC:papersize letter
> > /usr/local/share/groff/1.22.3/font/devlj4/DESC:papersize letter
> > /usr/local/share/groff/1.22.3/font/devps/DESC:papersize letter
> > /usr/local/share/groff/1.22.3/font/devlbp/DESC:papersize letter
> > /usr/local/share/groff/1.22.3/font/devpdf/DESC:papersize letter
> 
> That is automatically generated at GNU troff build time, controlled
> by files generated by autoconf, controlled by files generated by
> automake, controlled by files autogenerated by whatever (insert
> your favourite rabbit hole here).
> 
> In any case, the fact that groff defaults to "papersize letter" is
> the reason why mandoc(1) does the same.  Unless there are strong
> reasons to diverge, mandoc aims for compatibility with groff.

Yes, I figured out it was an option selected at compile time (curiously
in Slackware, being american, groff is compiled to use a4).

What moved me to test this on Linux is I remember using the
/etc/papersize file there.  But it seems groff and gv ignore that file
(I mean on linux).


> 
> Yours,
>   Ingo

Thank you Ingo.

Re: mandoc output paper size

[Walter Alejandro Iglesias]

Answering myself.

In article <a675001fecbb3...@server.roquesor.com> Walter Alejandro Iglesias 
<w...@roquesor.com> wrote:
> As a side note.  You made me realize of something I didn't notice when
> I migrated to openbsd; I have files generated with GNU roff that
> defaults to letter size.  This doesn't happen on Linux, I ignore why.

This is set in DESC config files.

$ grep -ER 'papersize (letter|a4)' /usr/local/share/groff/*
/usr/local/share/groff/1.22.3/font/devdvi/DESC:papersize letter
/usr/local/share/groff/1.22.3/font/devlj4/DESC:papersize letter
/usr/local/share/groff/1.22.3/font/devps/DESC:papersize letter
/usr/local/share/groff/1.22.3/font/devlbp/DESC:papersize letter
/usr/local/share/groff/1.22.3/font/devpdf/DESC:papersize letter

Re: mandoc output paper size

[Walter Alejandro Iglesias]

In article <20171026104155982590.bfb59...@talsever.com> Amelia A Lewis 
<amyz...@talsever.com> wrote:
> On Thu, 26 Oct 2017 16:14:36 +0200 (CEST), Walter Alejandro Iglesias 
> wrote:
> > In the ps file generated by mandoc you should have this line:
> > 
> >   %%DocumentMedia: Default 595 841 0 () ()
> > 
> > Where 595 841 correspond to A4.  If you set output paper to "letter"
> > that line will say:
> > 
> >   %%DocumentMedia: Default 612 790 0 () ()
> 
> So these measures are in points?

I took it from here:

https://www.gnu.org/software/gv/manual/gv.html#Paper-Keywords-and-paper-size-in-points


> 
> https://en.wikipedia.org/wiki/Point_(typography)
> 

Re: mandoc output paper size

[Walter Alejandro Iglesias]

In article <20171026122507.ga13...@www.stare.cz> Jan Stary  
wrote:
> On Oct 26 11:36:45, w...@roquesor.com wrote:
> > In article <20171026083919.ga38...@www.stare.cz> Jan Stary  
> > wrote:
> > > I am not sure whether man -Tpdf and man -Tps honour the paper size.
> > 
> > I think it does.
> > 
> > I don't have a printer at hand to verify it but if in the gv(1) menu
> > I select alternativelly A4 (or Letter) and Default
> 
> You can "select alternatively" whatever you want in the gv(1) window,
> but that don't make it so. My point is that files which really are A4
> just already say so in the gv(1) box, without "selecting it alternatively".

In the ps file generated by mandoc you should have this line:

  %%DocumentMedia: Default 595 841 0 () ()

Where 595 841 correspond to A4.  If you set output paper to "letter"
that line will say:

  %%DocumentMedia: Default 612 790 0 () ()



As a side note.  You made me realize of something I didn't notice when
I migrated to openbsd; I have files generated with GNU roff that
defaults to letter size.  This doesn't happen on Linux, I ignore why.


> 
> > I can see how the page get resized (or not)
> > depending on the 'ouput paper' man.conf setting.
> 
> Yes it does. But why does it say e.g. "y841x595" instead of A4?
> (Maybe "A4" is just a shorthand for that, I don't know).
> 
> Jan
> 
> 

Re: mandoc output paper size

[Walter Alejandro Iglesias]

In article <20171026083919.ga38...@www.stare.cz> Jan Stary  
wrote:
> I am not sure whether man -Tpdf and man -Tps honour the paper size.

I think it does.

I don't have a printer at hand to verify it but if in the gv(1) menu
I select alternativelly A4 (or Letter) and Default I can see how the
page get resized (or not) depending on the 'ouput paper' man.conf
setting.


Walter

Re: SSH: lost connection after restarting pf. [SOLVED]

[Walter Alejandro Iglesias]

On Fri, Aug 18, 2017 at 07:31:05PM +0200, Otto Moerbeek wrote:
> On Sat, Aug 12, 2017 at 02:40:41PM +0200, Walter Alejandro Iglesias wrote:
> 
> > In article <20170812123632.p7zgt2l4kz43y...@symphytum.spacehopper.org> you 
> > wrote:
> > > On 2017/08/12 14:33, Walter Alejandro Iglesias wrote:
> > > > In article <5127ac707aa6f...@server.roquesor.com> you wrote:
> > > > > Hi Stuart,
> > > > > 
> > > > > In article <slrnootn18.31bc@naiad.spacehopper.org> you wrote:
> > > > > > On 2017-08-12, Walter Alejandro Iglesias <w...@roquesor.com> wrote:
> > > > > > > Yesterday while copying a big file from one machine to another in 
> > > > > > > my LAN
> > > > > > > I noticed that restarting pf:
> > > > > > >
> > > > > > >   # pfctl -d && pfctl -e -f /etc/pf.conf
> > > > > > >
> > > > > > > scp stops and quits showing this message:
> > > > > > >
> > > > > > >   - stalled - Conection reset by 192.168.1.*  Lost connection
> > > > > > >
> > > > > > >
> > > > > > > Is this expected or is a bug?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > 
> > > > > > Expected.
> > > > > > 
> > > > > > PF is a state-inspecting firewall and verifies things like TCP 
> > > > > > sequence
> > > > > > numbers; it needs to see the initial connection handshake to pick 
> > > > > > up the
> > > > > > wscale value.
> > > > > > 
> > > > > > I would recommend just reloading the ruleset rather than disabling 
> > > > > > and
> > > > > > re-enabling PF first.
> > > > > > 
> > > > > > 
> > > > > 
> > > > > I have this rule:
> > > > > 
> > > > > block in log quick inet proto tcp from  to port ssh
> > > > > 
> > > > > That reads IPs from a the "port22" file which is updated from a script
> > > > > in a cronjob.  I ignore which command to use to re-read that file
> > > > > without causing the interrupt.
> > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > You mean doing only this?
> > > > 
> > > > # pfctl -f /etc/pf.conf
> > > 
> > > Yes.
> > > 
> > > 
> > 
> > I just tried it and works OK.  Thank you very much.
> > 
> 
> A bit reply late due to vacation...
> 
> I would like to stress that disable and then a reload is a
> dangerous practise. Apart from the fact that it looses state it also
> will leave pf disabled if you made a syntax error in your ruleset.

Yes, I was worried about that.

> 
> Please just do a reload: it is much more safer: it will first
> validate the new ruleset and then *atomically* replace the old with
> the new ruleset, leaving intact any relevant state information.

I don't remember exactly what made me think that in the specific case of
tables reading IP lists from files a reload wasn't enough.  Something
wrong I did while testing lead me to wrong conclusions. :-)


Thank you!


> 
>   -Otto


Walter

Re: Mastering opensmtpd rules

[Walter Alejandro Iglesias]

On Tue, Aug 15, 2017 at 05:10:00PM +0200, Gilles Chehade wrote:
> On Tue, Aug 15, 2017 at 01:29:16PM +0200, Walter Alejandro Iglesias wrote:
> > > 
> > >   accept from any for any virtual  [...]
> > > 
> > 
> > Besides, after modifying that rule in the file I also had to change the
> > order.  Since rules below the "catch-all" one never get evaluated, it
> > has forcibly to be the last one:
> > 
> >[...]
> >accept from local for local alias  deliver to mbox
> >accept from local sender  for any relay
> >accept from any for any virtual  deliver to mbox
> ># End of file
> > 
> 
> Not a truth written in stone but, usually, having the "from any for any"
> rule in a config file is a sign that user failed to write ruleset and is
> using this as a fallback.

The word "mastering" I used in the subject may lead to confusion.  I
should've written "starting with" instead. :-)

My smtpd.conf is not a finished work.  Step by step.

> The earliest the rules match the envelope, the
> better, as it indicates that the rule was written to match precisely.
> 

My intention was to find the way to support the "postmaster" address,
that RFC requires to be supported even *with no domain specification.*
I wasn't able to figure out how to solve this while the "domain" table
was included in the rule.  Without that table now I can add to the
"valiases" file this:

postmaster  myuser
s...@site1.com  ...
s...@site2.com  ...

To make available any of this addresses:

postmaster@[IP_ADDRESS]
postmas...@site1.com
postmas...@site2.com

> Most rulesets should finish with a relay (via?) rule from local for any.

That's the way I had it, but I couldn't send mail when preceded by "from
any to any" rule.  I know my current solution is sloppy, I'll try to
study a bit more and improve my configuration.  Thank you for your help.

> 
> 
> -- 
> Gilles Chehade
> 
> https://www.poolp.org  @poolpOrg

Re: Mastering opensmtpd rules

[Walter Alejandro Iglesias]

> 
>   accept from any for any virtual  [...]
> 

Besides, after modifying that rule in the file I also had to change the
order.  Since rules below the "catch-all" one never get evaluated, it
has forcibly to be the last one:

   [...]
   accept from local for local alias  deliver to mbox
   accept from local sender  for any relay
   accept from any for any virtual  deliver to mbox
   # End of file

Re: Mastering opensmtpd rules

[Walter Alejandro Iglesias]

Hi Gilles,

On Tue, Aug 15, 2017 at 11:15:32AM +0200, Gilles Chehade wrote:
> On Tue, Aug 15, 2017 at 09:22:41AM +0200, Walter Alejandro Iglesias wrote:
> > Hello everyone,
> > 
> > I'd appreciate experienced opensmtpd users tell me if I'm understanding
> > well the mechanism in the following rule.
> > 
> > Currently, in my smtpd.conf I have this line:
> > 
> >   accept from any for domain  virtual  deliver to mbox
> > 
> > But since all keys in my "valiases" table are full email addresses, in
> > the form:
> > 
> >   u...@example.org  user
> > 
> > I'm thinking the use of "vdomains" table is redundant.  I could safely
> > simplify the rule to:
> > 
> >   accept from any for any virtual  deliver to mbox
> > 
> > 
> > Am I wrong in this assumption?
> >
> 
> kind of, smtpd.conf being a first match ruleset it is impossible to make
> this kind of analysis without having your other rules too.

Sorry, I should've added it's the only "from any" rule I have:


# /etc/mail/smptd.conf

egress_int="em0"
server="server.roquesor.com"

table aliases file:/etc/mail/aliases
table valiases file:/etc/mail/valiases
table vdomains file:/etc/mail/vdomains
table addresses file:/etc/mail/addresses
table users file:/etc/mail/users

pki $server certificate "/etc/ssl/server.crt"
pki $server key "/etc/ssl/private/server.key"

listen on lo0
listen on $egress_int port 25 tls pki $server
listen on $egress_int port 465 smtps pki $server auth \
senders  masquerade

accept from local for local alias  deliver to mbox
accept from any for domain  virtual  deliver to mbox
accept from local sender  for any relay

# End of file


> 
> in this case, this may or may not give the desired behavior depending on
> rules following it because envelope matching happens _before_ virtual is
> even evaluated.
> 
> with:
> 
> accept from any for domain  [...]
> 
> you will only match envelopes for the domains in , it allows a
> different rule to match other domains:
> 
> accept from any for domain  [...]
> accept from any for domain foobar.org [...]
> 
> with:
> 
> accept from any for any [...]
> 
> you will match all envelopes so you're essentially creating a catch-all.
> 
> 
> virtual happens AFTER a rule has been matched so if you recipient is not
> found the RCPT will be rejected, smtpd will not search for another rule.

If I'm understanding you well then it's what I want.

My question was if the "virtual" entry in the rule is enough to reject
not matching recipients.  For example, having this rule:

  accept from any for any virtual  [...]

and a "valiases" file containing only this line:

  l...@foobar.org   user

will messages sent to i.e. l...@foobar2.org or l...@foobar3.org be
rejected?



> 
> 
> -- 
> Gilles Chehade
> 
> https://www.poolp.org  @poolpOrg

Mastering opensmtpd rules

[Walter Alejandro Iglesias]

Hello everyone,

I'd appreciate experienced opensmtpd users tell me if I'm understanding
well the mechanism in the following rule.

Currently, in my smtpd.conf I have this line:

  accept from any for domain  virtual  deliver to mbox

But since all keys in my "valiases" table are full email addresses, in
the form:

  u...@example.org  user

I'm thinking the use of "vdomains" table is redundant.  I could safely
simplify the rule to:

  accept from any for any virtual  deliver to mbox


Am I wrong in this assumption?

Re: SSH: lost connection after restarting pf. [SOLVED]

[Walter Alejandro Iglesias]

In article <20170812123632.p7zgt2l4kz43y...@symphytum.spacehopper.org> you 
wrote:
> On 2017/08/12 14:33, Walter Alejandro Iglesias wrote:
> > In article <5127ac707aa6f...@server.roquesor.com> you wrote:
> > > Hi Stuart,
> > > 
> > > In article <slrnootn18.31bc@naiad.spacehopper.org> you wrote:
> > > > On 2017-08-12, Walter Alejandro Iglesias <w...@roquesor.com> wrote:
> > > > > Yesterday while copying a big file from one machine to another in my 
> > > > > LAN
> > > > > I noticed that restarting pf:
> > > > >
> > > > >   # pfctl -d && pfctl -e -f /etc/pf.conf
> > > > >
> > > > > scp stops and quits showing this message:
> > > > >
> > > > >   - stalled - Conection reset by 192.168.1.*  Lost connection
> > > > >
> > > > >
> > > > > Is this expected or is a bug?
> > > > >
> > > > >
> > > > >
> > > > 
> > > > Expected.
> > > > 
> > > > PF is a state-inspecting firewall and verifies things like TCP sequence
> > > > numbers; it needs to see the initial connection handshake to pick up the
> > > > wscale value.
> > > > 
> > > > I would recommend just reloading the ruleset rather than disabling and
> > > > re-enabling PF first.
> > > > 
> > > > 
> > > 
> > > I have this rule:
> > > 
> > > block in log quick inet proto tcp from  to port ssh
> > > 
> > > That reads IPs from a the "port22" file which is updated from a script
> > > in a cronjob.  I ignore which command to use to re-read that file
> > > without causing the interrupt.
> > > 
> > > 
> > > 
> > 
> > You mean doing only this?
> > 
> > # pfctl -f /etc/pf.conf
> 
> Yes.
> 
> 

I just tried it and works OK.  Thank you very much.

Re: SSH: lost connection after restarting pf.

[Walter Alejandro Iglesias]

In article <5127ac707aa6f...@server.roquesor.com> you wrote:
> Hi Stuart,
> 
> In article <slrnootn18.31bc@naiad.spacehopper.org> you wrote:
> > On 2017-08-12, Walter Alejandro Iglesias <w...@roquesor.com> wrote:
> > > Yesterday while copying a big file from one machine to another in my LAN
> > > I noticed that restarting pf:
> > >
> > >   # pfctl -d && pfctl -e -f /etc/pf.conf
> > >
> > > scp stops and quits showing this message:
> > >
> > >   - stalled - Conection reset by 192.168.1.*  Lost connection
> > >
> > >
> > > Is this expected or is a bug?
> > >
> > >
> > >
> > 
> > Expected.
> > 
> > PF is a state-inspecting firewall and verifies things like TCP sequence
> > numbers; it needs to see the initial connection handshake to pick up the
> > wscale value.
> > 
> > I would recommend just reloading the ruleset rather than disabling and
> > re-enabling PF first.
> > 
> > 
> 
> I have this rule:
> 
> block in log quick inet proto tcp from  to port ssh
> 
> That reads IPs from a the "port22" file which is updated from a script
> in a cronjob.  I ignore which command to use to re-read that file
> without causing the interrupt.
> 
> 
> 

You mean doing only this?

# pfctl -f /etc/pf.conf

Re: SSH: lost connection after restarting pf.

[Walter Alejandro Iglesias]

Hi Stuart,

In article <slrnootn18.31bc@naiad.spacehopper.org> you wrote:
> On 2017-08-12, Walter Alejandro Iglesias <w...@roquesor.com> wrote:
> > Yesterday while copying a big file from one machine to another in my LAN
> > I noticed that restarting pf:
> >
> >   # pfctl -d && pfctl -e -f /etc/pf.conf
> >
> > scp stops and quits showing this message:
> >
> >   - stalled - Conection reset by 192.168.1.*  Lost connection
> >
> >
> > Is this expected or is a bug?
> >
> >
> >
> 
> Expected.
> 
> PF is a state-inspecting firewall and verifies things like TCP sequence
> numbers; it needs to see the initial connection handshake to pick up the
> wscale value.
> 
> I would recommend just reloading the ruleset rather than disabling and
> re-enabling PF first.
> 
> 

I have this rule:

block in log quick inet proto tcp from  to port ssh

That reads IPs from a the "port22" file which is updated from a script
in a cronjob.  I ignore which command to use to re-read that file
without causing the interrupt.

Re: SSH: lost connection after restarting pf.

[Walter Alejandro Iglesias]

On Sat, Aug 12, 2017 at 11:08:23AM +0200, Walter Alejandro Iglesias wrote:
> Yesterday while copying a big file from one machine to another in my LAN
> I noticed that restarting pf:
> 
>   # pfctl -d && pfctl -e -f /etc/pf.conf

I assume it's not necessary to say I'm doing this without changing any
rule on pf.conf. :-)


> 
> scp stops and quits showing this message:
> 
>   - stalled - Conection reset by 192.168.1.*  Lost connection
> 
> 
> Is this expected or is a bug?
> 
> 

SSH: lost connection after restarting pf.

[Walter Alejandro Iglesias]

Yesterday while copying a big file from one machine to another in my LAN
I noticed that restarting pf:

  # pfctl -d && pfctl -e -f /etc/pf.conf

scp stops and quits showing this message:

  - stalled - Conection reset by 192.168.1.*  Lost connection


Is this expected or is a bug?

Re: New question, do I really need a AAAA record?

[Walter Alejandro Iglesias]

On Thu, Aug 10, 2017 at 07:26:16PM +0100, Stuart Henderson wrote:
> Rephrasing: if you make an outgoing SMTP connection, a reverse DNS PTR
> record should exist for the source address you're connecting from (whether
> that's v4 or v6), and an A (for v4) or  (for v6) lookup for the name
> in that PTR should give back the same address.
> 
> For your example:
> 
> 185.37.212.61 -> server.roquesor.com
> server.roquesor.com -> 185.37.212.61
> 
> That looks good.
> 
> If you are making outgoing SMTP connections from a v6 address, then you
> should have matching PTR+ as well.

OK.  Huff!, it's just I got dizzy.  After all the advises I got in
this thread I started to doubt even about my existence. :-)


Thank you Stuart!

Re: New question, do I really need a AAAA record?

[Walter Alejandro Iglesias]

In article  you wrote:
> Hi Stuart,
> 
> In article  you wrote:
> > On 2017-08-10, Rui Ribeiro  wrote:
> > > An email server in a residential setting will fail PTR unless you are
> > > working with a medium sized/an ISP that cares about their customers.
> > >
> > > see answer here
> > > https://unix.stackexchange.com/questions/371329/bind-proper-reverse-config
> > 
> > You can't expect to reliably deliver email unless you have a PTR record and
> > an A/ record (at least within the same domain, though in some cases
> > the full hostname needs to match).
> > 
> 
> At this point things got a bit confusing.  First of all I don't run my
> own DNS server, I use the free dns service from the registrar company
> where I bought my domain names.  There I configured the records I need
> for the web and mail servers I run at home.  Then, asking my ISP to add
> a PTR record on *their* DNS was the first thing I did when I contracted
> the service, and was the first thing I checked again last weekend after
> the problem I explain in this thread happened.  Despite the negative
> results the website someone recommended me shows (dnsinspect.com) I
> think my PTR is working well, you can use host(1), dig(1) or nslookup(1)
> to check my IP (185.37.212.61) against yours or any public DNS to
> corroborate it.  Or simply put the IP in your browser URL bar, press
> ENTER and see if it resolves to my web site. :-)
> 
> Stated the above, now the new question.  By A/ records I understand
> you mean the records on *my* side (not my ISP's), don't you?  Well,
> since I'm not using ipv6 I didn't added any  record.  Do you
> recommend me to add it, anyways?
> 
> 

Sorry, I think I didn't formulate the question well.  What I meant was,
do I need also a static ipv6 to be considered by big smtp servers as a
legal sender?

New question, do I really need a AAAA record?

[Walter Alejandro Iglesias]

Hi Stuart,

In article  you wrote:
> On 2017-08-10, Rui Ribeiro  wrote:
> > An email server in a residential setting will fail PTR unless you are
> > working with a medium sized/an ISP that cares about their customers.
> >
> > see answer here
> > https://unix.stackexchange.com/questions/371329/bind-proper-reverse-config
> 
> You can't expect to reliably deliver email unless you have a PTR record and
> an A/ record (at least within the same domain, though in some cases
> the full hostname needs to match).
> 

At this point things got a bit confusing.  First of all I don't run my
own DNS server, I use the free dns service from the registrar company
where I bought my domain names.  There I configured the records I need
for the web and mail servers I run at home.  Then, asking my ISP to add
a PTR record on *their* DNS was the first thing I did when I contracted
the service, and was the first thing I checked again last weekend after
the problem I explain in this thread happened.  Despite the negative
results the website someone recommended me shows (dnsinspect.com) I
think my PTR is working well, you can use host(1), dig(1) or nslookup(1)
to check my IP (185.37.212.61) against yours or any public DNS to
corroborate it.  Or simply put the IP in your browser URL bar, press
ENTER and see if it resolves to my web site. :-)

Stated the above, now the new question.  By A/ records I understand
you mean the records on *my* side (not my ISP's), don't you?  Well,
since I'm not using ipv6 I didn't added any  record.  Do you
recommend me to add it, anyways?

Re: gmail and hotmail blocking mail sent from my IP

[Walter Alejandro Iglesias]

Hello Rupert,

In article 

How about to let this die?

[Walter Alejandro Iglesias]

Guys,

The issue was solved after the fist answer (Martijn van Duren's).

Everyone's opinions have been very useful.  But since this is not
OpenBSD related I propose to let it die.

Re: gmail and hotmail blocking mail sent from my IP

[Walter Alejandro Iglesias]

In article <20170808121343.46a8ddb9@fir.internal> you wrote:
> Hi Walter:
> 
> On Sun, 6 Aug 2017 19:45:22 +0200 Walter Alejandro Iglesias wrote:
> > What determines those "ranges", who regulates that?
> 
> Some ISPs submit IP blocks to various blacklists. e.g:
> https://www.Spamhaus.Org/faq/section/Spamhaus%20PBL#242
> http://www.Sorbs.Net/faq/dul.shtml
> 
> Asking your ISP to exclude your addresses might help.


I sent an email to my ISP, they don't even know about this lists. :-)

Besides, I sent an email to spamhaus.org suggesting them not to include
static IPs in their PBL list by default as they do.


I'll take this chance to share my thinking with everyone here.

I understand that given everyone uses gmail, hotmail or mail provided by
some multinational hosting service they assume mail coming from
residential connections cannot be other thing but spam sent from hacked
machines.  But someone paying for a static IP in a residential
connection is the opposite case.  When you have to deal with thousands
of users you resort to any trick you find on the Internet and start to
blindly blacklist all; this is a big servers problem.  And the more
users you have to deal with the worse.  On the contrary, from my part, I
have just a pair of personal addresses, so it's not a big deal for me to
audit my server and use more sane, less harmful and, overall, more
effective measures to filter spam and to prevent spam be sent from my
machine.  And I think this is the direction everyone should point to
instead of resting day after day more and more on big companies for
everything.  In general, everyone should tend to decentralize instead of
monopolize.  The real problem is the passive attitude most people assume
in the use of the Internet (and life in general but I don't want to bore
you with cheap philosophy. :-))


> 
> Regards,


Thank you for your advice.

Re: gmail and hotmail blocking mail sent from my IP

[Walter Alejandro Iglesias]

In article <slrnooes63.31bc@naiad.spacehopper.org> you wrote:
> On 2017-08-06, Walter Alejandro Iglesias <w...@roquesor.com> wrote:
> > I visited spamhaus.org site and found out my IP is included in a list
> > called PBL that, as they  explain is not a spammers list, it just
> > includes dynamic and "non mail server IP ranges".
> >
> > Does someone here know what is "non mail server IP ranges" about?  Or,
> > how could my static IP could be taken as dynamic (some DNS faliure at my
> > ISP end?).
> 
> You should add ypur IP to dnswl.org. I can't guarantes it will help
> everywhere, but it will help some places and won't hurt others.
> 
> 

I'll take a look to dnswl.org.  Thanks.

It seems that after removing my IP from spamhaus pbl the issue is fixed.

Re: gmail and hotmail blocking mail sent from my IP

[Walter Alejandro Iglesias]

Hi Niels,

On Sun, Aug 06, 2017 at 07:19:04PM +0200, Niels Kobschätzki wrote:
> 
> > On 6. Aug 2017, at 18:40, Walter Alejandro Iglesias <w...@roquesor.com> 
> > wrote:
> > 
> >> On Sun, Aug 06, 2017 at 06:02:25PM +0200, Jesper Wallin wrote:
> >> Like Martijn pointed out, you're sending mail from a IP which is not
> >> intended for mail-servers.
> > 
> > This was my main question.  What is an "IP intended for mail-servers"?
> 
> The question should be "what are IPs **not** intended for mail-servers?"
> 
> The ranges of ISPs for home-users and the dsl-, cable-, whatever-connection 
> are well-known and pretty much on all of the blacklists since the only thing 
> you can usually expect from them is spam from botnets. Legitimate mails are 
> rather rare from those ranges, thus they get blocked. 

I cannot tell what happens in pratice, I've never run a big mail server.
But the reasons that come to my mind someone wants to run their own
server (at home or at a small enterprise) are opposed to what you state.
Why would you want to send spam from the fixed IP you're paying for (in
my case 5 euros mouth)?

The question is still unanswered.  What determines those "ranges", who
regulates that?


> To not get blocked by google and hotmail you need an IP from some
> hosting-provider, university or something like this;

Which is the procedure followed by those entities to get an IP in what
you called the "authorized range"?  Authorized by who?


> a PTR-record for your server

I already have this.


> and at least an SPF-, even better a DKIM-record.

I had these at first and removed them after seeing they don't help.


> And if you
> ever send out mail, you maybe want a secondary IP for temporary
> failover-cases if you land  temporarily on a black list.

I have just two personal addresses.  I don't need that complication.  :-)



> 
> Niels

Re: gmail and hotmail blocking mail sent from my IP

[Walter Alejandro Iglesias]

On Sun, Aug 06, 2017 at 06:02:25PM +0200, Jesper Wallin wrote:
> Like Martijn pointed out, you're sending mail from a IP which is not
> intended for mail-servers.

This was my main question.  What is an "IP intended for mail-servers"?

Re: gmail and hotmail blocking mail sent from my IP

[Walter Alejandro Iglesias]

Hi Gareth,

On Sun, Aug 06, 2017 at 04:12:45PM +0100, Gareth Nelson wrote:
> I'm assuming that you have your SPF records setup correctly.
> 

I did that at first, and all the tricks (dkim, etc) they ask to make you
appear as a legal sender, but after confirming my mail still went to
SPAM in both (gmail, hotmail) I remove all that trickery.

Re: gmail and hotmail blocking mail sent from my IP

[Walter Alejandro Iglesias]

Hi Martijn,

On Sun, Aug 06, 2017 at 05:09:10PM +0200, Martijn van Duren wrote:
> Not an authority on this, so take my reply for what you want.
> 
> As far as I know this list is used to keep track of ip-addresses by ISPs
> for home-addresses, which are not intended to be used for outgoing mail.
> 
> You can whitelist your ip-address on this list yourself and all should
> be back to normal.

I just did it from spamhause site.

> 
> I faced the same issues and adding my ip did solve the 550s.
> 
> Do note that my ip gets removed every year and thus should be re-added
> ever year.

I'll take this in care.  Thank you!



> 
> Sincerely,
> 
> martijn@
> 

gmail and hotmail blocking mail sent from my IP

[Walter Alejandro Iglesias]

Hello everyone,

I was using smtpd(8) (static IP and FQDN resolving direct and reverse)
for a year without problems.  Today sending from my server (from the
same address I'm using now) to gmail and hotmail they answered the
following (MAILER-DAEMON answer).

Sending to gmail addresses:

  *@gmail.com: 550-5.7.1 [185.37.212.61] The IP you're using to send
  mail is not authorized to send email directly to our servers.  Please
  use the SMTP relay at your service provider instead. Learn more at
  https://support.google.com/mail/?p=NotAuthorizedError
  e1si6736354wra.236 - gsmtp

Sending to hotmail:

  *@hotmail.com: 550 DY-001 (SNT004-MC3F42) Unfortunately, messages from
  185.37.212.61 weren't sent. Please contact your Internet service
  provider. You can tell them that Hotmail does not relay
  dynamically-assigned IP ranges. You can also refer your provider to
  http://mail.live.com/mail/troubleshooting.aspx#errors.


On the hotmail link above the explanaition for code DY-001 is:

  Mail rejected by Outlook.com for policy reasons. We generally do not
  accept email from dynamic IP's as they are not typically used to
  deliver unauthenticated SMTP email to an Internet mail server. If you
  are not an email/network admin please contact your Email/Internet
  Service Provider for help. http://www.spamhaus.org maintains lists of
  dynamic and residential IP addresses.

It doesn't happen with yahoo.

I visited spamhaus.org site and found out my IP is included in a list
called PBL that, as they  explain is not a spammers list, it just
includes dynamic and "non mail server IP ranges".

Does someone here know what is "non mail server IP ranges" about?  Or,
how could my static IP could be taken as dynamic (some DNS faliure at my
ISP end?).

Re: Helping out

[Walter Alejandro Iglesias]

Hello Bryan and Radoslav,

In article <20170802015654.ga64...@c.brycv.com> you wrote:
> On Tue, Aug 01, 2017 at 08:19:23PM -0400, Radoslav_Mirza wrote:
> > Dear Group, Are there any places to start helping out for a beginner?
> > Any junior jobs or todo lists?
> > 
> > I have a new Ryzen 1700 running OpenBSD so maybe I could help with
> > some benchmark tests etc.
> > 
> > Any pointers of where to go would be great!
> 
> There was a recent discussion about ProtonMail not sending plain text
> email which this list expects. I would suggest sending with another
> address and sending in plain text. Check the archives for more info
> about it but base64 encoded emails (like from ProtonMail) will likely be
> ignored. Hopefully ProtonMail will correct this problem but they have
> "started" on it for more than a year.

The first time I looked at the base64 encoded text pasted by Mihai
Popescu's (the first noticing this issue):

https://marc.info/?l=openbsd-misc=149984510728808=2

I saw the message was written in English, what made me think protonmail
was doing something wrong, but more late I realized I'd overlooked the
first line, the quoted text reference author's name contained *one*
non-ascii character. :-)

To see it yourself:

$ cat file-containing-only-base64-part | openssl enc -base64 -d

This means what proton mail did in this case isn't incorrect.

As far as I understand, the purpose of this encoding (as the whole MIME
standard) is to send all messages through the net in plain ascii, to
assure compatibility among all servers.  For example if I typed here any
non ascii character (what could happen even by accident when you use a
non English keyboard), Mutt, the MUA I use, would send the body of this
message quoted-printable encoded (the one used for low utf8 density
languages as Spanish; base64 is used i.e. for Russian).  The same would
happen if some non-ascii character is in some sender's name in the
quoted text references; your MUA would detect that character and
automatically would send the body of your message encoded.  Despite
base64, quoted-printable would still be readable.

Where is the problem.  I guess developers here, when they don't have any
MUA from packages installed, are forced to use the one in base,
mailx(1), which doesn't support MIME.  If this is the case, they'd have
troubles reading non ascii characters sent as is anyway.  So, the best
workaround, whatever MUA you use, is to avoid using non-ascii characters
when you post to these lists (even in your name).

Said that I still find annoying top-posting and not hard wrapped lines.
But protonmail isn't the only one doing this. ;-)

(I'd add more common practices you can't blame MUAs as not using double
spaces after sentences, writing all in lowercase; the time they save
writing is charged to the reader.)



> 
> Bryan
> 
> 

Re: mime headers quoted-printable

[Walter Alejandro Iglesias]

Inspired in the new utf8 man page (thanks tedu@) I think I found a
solution to the charset issue.

New version:


/*
 * MIME encode mail header quoted-printable.
 *
 */

#include 

#define ASCII   0x7f
#define IN  1
#define OUT 0
#define MAX 1024

int
main()
{
int c, i, n, nl, count, isutf8, eightbit, encode, body;
unsigned char s[MAX];

i = n = nl = count = 0;
encode = eightbit = body = isutf8 = OUT;

while ((c = getchar()) != EOF) {
if (body == IN)
putchar(c);
else if (c == '\n' || c == ' ') {
s[i] = '\0';
if (eightbit == IN) {
if (encode == IN)
printf("=20?= ");
if (isutf8 == OUT)
printf("=?ISO-8859-1?Q?");
else
printf("=?UTF-8?Q?");

while (n < i) {
if (s[n] > ASCII ||
s[n] == '=' ||
s[n] == '?' ||
s[n] == '\t')
printf("=%02X", s[n++]);
else
printf("%c", s[n++]);
}
n = 0;
encode = IN;

if (c == '\n') {
printf("?=");
putchar(c);
encode = OUT;
}

eightbit = OUT;
} else {
if (encode == IN)
printf("?= ");
printf("%s", s);
putchar(c);
encode = OUT;
}
i = 0;
if (c == '\n')
++nl;
else
nl = 0;
if (nl > 1)
body = IN;
} else {
if (c > ASCII) {
eightbit = IN;
++count;
if (count == 1) {
if (c != 0xc2 &&
c != 0xc3 &&
c != 0xe2)
isutf8 = OUT;
else
isutf8 = IN;
}
} else
count = 0;
if (i >= MAX - 1) {
s[i] = '\0';
printf("%s", s);
i = 0;
}
s[i++] = c;
}
}

return 0;
}

Re: mime headers quoted-printable

[Walter Alejandro Iglesias]

An overlook I can't figure out why didn't core dumped.

--- encode-qprint-header.c  Wed May 24 22:04:24 2017
+++ encode-qprint-header.c  Wed May 24 22:03:49 2017
@@ -66,13 +66,12 @@ main()
} else {
if (c > ASCII)
eightbit = IN;
-   if (i >= MAX) {
+   if (i >= MAX - 1) {
s[i] = '\0';
printf("%s", s);
i = 0;
}
s[i] = c;
-
++i;
}
}

Re: mime headers quoted-printable

[Walter Alejandro Iglesias]

I was pointed out words (no spaces) longer than 256 characters produce a
buffer overflow with my previous version.

I scanned  my saved (since ~ 2005) mbox for header lines without spaces
longer than 256 and found several.  Most of them are non wrapped base64
encoded text, a few are "References:" separated with commas instead of
spaces.  So I think I can just ignore those lines.

Besides that, the new version below has a limit.  No more buffer
overflow.

By the way, I can make it wrap lines bigger than 76 columns as the
standards ask, but looking at mailx code I observed it already takes
care of that.


/*
 * MIME encode mail header quoted-printable. (VERSION 2)
 *
 * BUG: it assumes all non ascii characters are UTF-8.
 */

#include 

#define ASCII   0x7f
#define IN  1
#define OUT 0
#define MAX 256

int
main()
{
int c, i, n, nl, eightbit, encode, body;
unsigned char s[MAX];

i = n = nl = 0;
encode = eightbit = body = OUT;

while ((c = getchar()) != EOF) {
if (body == IN)
putchar(c);
else if (c == '\n' || c == ' ') {
s[i] = '\0';
if (eightbit == IN) {
if (encode == IN)
printf("=20?= ");
printf("=?UTF-8?Q?");
while (n < i) {
if (s[n] > ASCII ||
s[n] == '=' ||
s[n] == '?' ||
s[n] == '\t') {
printf("=%02X", s[n]);
} else
printf("%c", s[n]);
++n;
}
n = 0;
encode = IN;

if (c == '\n') {
printf("?=");
putchar(c);
encode = OUT;
}

eightbit = OUT;
} else {
if (encode == IN)
printf("?= ");
printf("%s", s);
putchar(c);
encode = OUT;
}
i = 0;
if (c == '\n')
++nl;
else
nl = 0;
if (nl > 1)
body = IN;
} else {
if (c > ASCII)
eightbit = IN;
if (i >= MAX) {
s[i] = '\0';
printf("%s", s);
i = 0;
}
s[i] = c;

++i;
}
}

return 0;
}

mime headers quoted-printable

[Walter Alejandro Iglesias]

Hello everyone,

With mailx(1) in mind and resurrecting the few I know about C I wrote
the code pasted below.  It encodes mail headers in MIME quoted-printable
format.  Unless I'm missing something it complies with all stated here:

https://www.ietf.org/rfc/rfc2047.txt

You can pipe to it a line or the whole meassage, it only processes
headers leaving the body untouched.  The problem is, so far, it assumes
all 8bit chars are utf8.  Even when openbsd now only supports utf8
locale you can still enter iso-latin characters while your LC_CTYPE is
set to C, what means if eventually you judge it could be adapted to
patch mailx some non valid utf8 characters check would be mandatory.

So at this point I'm not sure it'll be a good deal since, as far as I
know, to make it able to check for non valid UTF-8 characters would take
more lines of code than the program itself.

I wrote also an encoder to the body, a base64 version and some decoders
but for now I'll show you only this one to not clutter the message.

I'd appreciate your opinion and advice about what can I do from now
(don't hesitate in being frank if you think it's useless).


/*
 * MIME encode mail headers quoted-printable.
 *
 * BUG: it assumes all non ascii characters are UTF-8.
 */

#include 

#define ASCII   0x7f
#define IN  1
#define OUT 0

int
main()
{
int c, i, n, nl, eightbit, encode, body;
unsigned char s[256];

i = n = nl = 0;
encode = eightbit = body = OUT;

while ((c = getchar()) != EOF) {
if (body == IN)
putchar(c);
else if (c == '\n' || c == ' ') {
s[i] = '\0';
if (eightbit == IN) {
if (encode == IN)
printf("=20?= ");
printf("=?UTF-8?Q?");
while (n < i) {
if (s[n] > ASCII ||
s[n] == '=' ||
s[n] == '?' ||
s[n] == '\t') {
printf("=%02X", s[n]);
} else
printf("%c", s[n]);
++n;
}
n = 0;
encode = IN;

if (c == '\n') {
printf("?=");
putchar(c);
encode = OUT;
}

eightbit = OUT;
} else {
if (encode == IN)
printf("?= ");
printf("%s", s);
putchar(c);
encode = OUT;
}
i = 0;
if (c == '\n')
++nl;
else
nl = 0;
if (nl > 1)
body = IN;
} else {
if (c > ASCII)
eightbit = IN;
s[i] = c;
++i;
}
}

return 0;
}

Re: OpenBSD 6.1: httpd.conf macro usage and string concatenation

[Walter Alejandro Iglesias]

In article <39c822f4-07f1-3544-0a8e-b75446f94...@4ss.de> you wrote:
> Hi!
> 
> I thought I could copy the same static server definition block and only
> change a unique macro definition at the top of each server. But this is
> not working:
> 
> ##
> # from httpd.conf
> ##
> # [...]
> 
> # macro definition
> certroot="/etc/ssl/httpd"
> docroot="/htdocs"
> 
> domain="domain.tld"
> server $domain{
>  listen on * tls port 443
>  tls certificate $certroot/$domain/$domain.pem
>  tls key $certroot/$domain/$domain.key
>  root $docroot/$domain
> }
> 
> domain="anotherdomain.tld"
> server $domain{
>  listen on * tls port 443
>  tls certificate $certroot/$domain/$domain.pem
>  tls key $certroot/$domain/$domain.key
>  root $docroot/$domain
> }
> 
> # [...]
> ##
> 
> The idea was if you have a lot of server definitions you could keep
> static the parts that are the same and just change the macro for each
> server the line above the server block.
> 
> Because httpd.conf man page says "Macros are not expanded inside
> quotes." I cannot use 'root "$docroot/$domain"'. But 'root
> $docroot/$domain' isn't accepted either. Does that mean I cannot use
> Macros for parts of the config file that reference to files or folders,
> because Macros are not expanded inside quotes but keywords with file or
> folder options require enclosing quotes? If that's the case I don't
> understand what Macros are good for.
> 
> Thanks in advance!
> 
> T.
> 
> 

There is another problem I mentioned here time ago.  Macros have
problems with slashes.  The same happens in pf.conf (and perhaps with
smtpd.conf too).  Unless it was already fixed, when you want to add a
path to a macro you must enclose it within double and single quotes:

certroot='"/etc/ssl/httpd"'

Then $certroot is expanded to "/etc/ssl/httpd" including the doble
quotes.