Re: svn commit: r326225 - head/sys/arm64/arm64

[Bruce Evans]

On Sun, 26 Nov 2017, Andrew Turner wrote:


Log:
 Make the arm64 pmap_invalidate functions static inline. This fixes building
 with DIAGNOSTIC.


This is fairly broken on x86 too.


Modified: head/sys/arm64/arm64/pmap.c
==
--- head/sys/arm64/arm64/pmap.c Sun Nov 26 06:31:34 2017(r326224)
+++ head/sys/arm64/arm64/pmap.c Sun Nov 26 09:29:34 2017(r326225)
@@ -896,7 +896,7 @@ SYSCTL_ULONG(_vm_pmap_l2, OID_AUTO, promotions, CTLFLA
/*
 * Invalidate a single TLB entry.
 */
-PMAP_INLINE void
+static __inline void
pmap_invalidate_page(pmap_t pmap, vm_offset_t va)


PMAP_INLINE is now only used once on arm64 (for pmap_kremove).  It gives
a small optimization or pessimization by inlining, but is fragile and
broken in some cases.

In the DIAGNOSTIC case, it is usually or always defined as empty.  This is
silly.  It does nothing except insignificantly simplify debugging and
expose the 2 bugs fixed in this commit.  These bugs are:
- on arm64, the pmap_invalidate*() functions are not declared in a header
  file or used outside of pmap.c, but they were declared as gnu
  [implicit-extern] inline for the reachable !DIAGNOSTIC case.
- complier bug: compilers don't detect the error that extern functions
  are not prototyped before they are defined in the gnu [implicit-extern]
  inline case
The bugs are exposed by the DIAGNOSTIC case.  Then the functions are
plain implicit-extern.

There is also an unreachable !DIAGNOSTIC case for non-gcc compatible
compilers.  On x86 and arm64, this uses explicit-extern inline.  This
gives C99 extern inline, which is confusing and probably doesn't worked.
If it worked, then it might be used instead of forcing gnu implicit-extern
inline if possible.

Plain arm is quite different:

arm/pmap-v4.c defines PMAP_INLINE as plain implicit-extern __inline in the
!PMAP_DEBUG case.  But this macro is completely different than the one of
the same name in most other pmap files.  It is only used when preceded by
'static', and thus is just for debugging, while its normal use is for
inlining public functions.  Some static functions are inlined without this
obfuscation.  i386 had the same obfuscation and more in FreeBSD-4 (it also
used PMAP_INLINE for extern functions).

arm/pmap-v6.c is more like arm64/pmap.c.  It ifdefs on DIAGNOSTIC but gives
plain implicit-extern C99 __inline only for extern functions where arm64
gives gnu implicit-extern non-C99 inline.

There are also style bugs.  Most places use __inline, but arm64 uses inline.
amd64/pmap.c uses __inline except twice for PMAP_INLINE and once for
pmap_unwire_ptp().

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326125 - head/usr.sbin/kgmon

[Bruce Evans]

On Sun, 26 Nov 2017, Edward Tomasz Napierala wrote:


On 1124T0138, Bruce Evans wrote:

On Thu, 23 Nov 2017, Edward Tomasz Napierala wrote:


Log:
 Mark kgmon(8) obsolete, redirecting users to pmcstat(8).


It isn't obsolete.  pmcstat can't do at least full (non-statistical)
call graphs and high-resolution profiling.


[snip detailed explanation]

Thanks.  Based on your arguments (and on the fact that I've tried
again to use it, following instructions from the URL below[1], and
this time it actually worked), I think I'll revert this commit,
and just add an ".Xr pmcstat 8" instead.


Thanks.


1. 
http://lserinol.blogspot.co.uk/2009/01/freebsd-kernel-profiling-with-kgmon.html


This is very basic (only the about confinguration), but that makes it short
enough to put in a section 4 man page or just in kgdb(8).  I don't know of
any tutorial for how to use either kgmon or pmcstat.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326095 - head/usr.sbin/bsdinstall/scripts

[Bruce Evans]

On Fri, 24 Nov 2017, Ian Lepore wrote:


On Fri, 2017-11-24 at 22:25 +1100, Bruce Evans wrote:

On Thu, 23 Nov 2017, Devin Teske wrote:
[...]

ntpdate's man page claims this, but is wrong AFAIK.? It says that the
functionality of ntpdate is now available in ntpd(8) using -q.? However
ntpdate -q is far from having equivalent functionality.? According to both
testing of the old version and its current man page, it does the same slow
syncing as normal ntpd startup, and just exits after this and doesn't
daemonize while doing this.? With the old version, this step takes 35-40
seconds starting with the time already synced to withing a few microseconds
(by a previous invocation of ntpd), while ntpdate syncs (perhaps not so
well) with a server half the world away in about 1 second.


Ahh, the good ol' days, when ntpdate was fast by default. ?Not
anymore...

unicorn# time ntpdate ntp.hippie.lan
24 Nov 15:21:31 ntpdate[734]: adjust time server [...] offset -0.000123 sec
0.013u 0.006s 0:06.13 0.1%??192+420k 0+0io 0pf+0w

If you want the fast old sub-second behavior these days, you have to
add -p1. ?Or, better yet, use sntp -r .


The default of -p4 hasn't changed, but its speed has.  I get the following
times for ntpdate -q -pN:
- old ntpdate -p1 0.31 seconds  (my system -> US server ping latency 180ms)
- -p2 0.52
- -p3 0.83
- -p4 0.95 (default for -p)
- new ntpdate -p1 0.37 seconds  (freefall -> same US server)
- -p2 2.39
- -p3 4.36
- -p4 6.36 (default)
- old ntpdate -p8 0.10 (max for -p) (my system -> localhost ping lat 0.014 ms)
- new ntpdate -p8 fail  (freefall -> localhost ping lat 0.060 ms)
- old ntpdate -p8 0.10  (my LAN -> my system ping lat 0.120 ms)
- new ntpdate -pN same as US server (freefall -> freebsd server ping lat 80 ms)
- old ntpdate -p8 0.24  (my system -> ISP server ping lat 12 ms)

This shows that old ntpdate -pN takes approximately the ping latency times N.
New ntpdate takes that for N = 1; otherwise it takes almost 2 seconds for
each increment of N.  ktrace shows many sleeps of 100 msec between
sendto/recvfrom pairs.


I'm not sure where you're coming up with numbers like "35 seconds" for
ntpd to initially step the clock. ?The version we're currently
distributing in base takes the same 6-7 seconds as ntpdate (assuming
you've used 'iburst' in ntp.conf). ?That's true in the normal startup
case, or when doing ntpd -qG to mimic ntpdate.


This is for old ntpd [-q] with iburst maxpoll 6, to the ISP server.  To
the LAN server, ntpd -q takes the same 35 seconds.  Normal startup with
ntpd -N high takes about the same time.

The examples in /etc/defaults/rc.conf don't give a hint about the -p flag
for ntpdate or the -N flag for ntpd.  Low -p values are probably good
enough for ntpdate before ntpd.


If there is an ntpd.drift file, ntpd is essentially sync'd as soon as
it steps.


If the drift file is correct.

I do have a correct drift file, and the above times are with it.  With
a correct driftfile and ntpdate before ntpd, ntpd is essentially synced
as soon as it starts :-).  When calibrating it manually, I verify this
by killing it soon after it starts and observing drift using ntpdate -q.


If there is not, it does a clock step, then does 300 seconds
of frequency training during which the clock can drift pretty far off-
time. ?It used to be possible to shorten the frequency training
interval with the 'tinker stepout' command, but the ntpd folks
decoupled that (always inapproriately overloaded) behavior between
stepout interval and training interval. ?There is no longer any way to
control the training interval at all, which IMO is a serious regression
in ntpd (albeit noticed primarily by those of us who DO have an atomic
clock and get a microsecond-accurate measurement of frequency drift in
just 2 seconds).


Is there any use for ntp as a client if you have an atomic clock?  Just to
validate both it and ntpd?

Bruce___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326073 - head/usr.bin/systat

[Bruce Evans]

On Fri, 24 Nov 2017, Konstantin Belousov wrote:


On Sat, Nov 25, 2017 at 12:14:10AM +1100, Bruce Evans wrote:
* ...

Hmm, I couldn't find any arch with even a compile-time variable PAGE_SIZE.
It is currently just unportable in theory to use hard-coded PAGE_SIZE or
macros that use it.

sparc64 uses 8K, Itanium had compile-time variable page size supported by
hardware.


I checked ia64, but couldn't find it before.  Now I found it.  The
ifdef is on LOG2_PAGE_SIZE (default 13 for 8K-pages).  This gives a
compile-time constant for PAGE_SIZE.  ogetpagesize() and the sysctl
HW_PAGESIZE return this constant.  So it may vary with the kernel in
userland, and userland would be broken if it used PAGE_SIZE from the
host include files.

LOG2_PAGE_SIZE was a supported global option, but this doesn't work in
modules.  No options work in modules, but this one is more fundamental
than most.  So it was probably necessary to compile all modules after
editing the default in the source file.  opt_global.h is even more
unusable in userland, but editing the source file works for userland too.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326073 - head/usr.bin/systat

[Bruce Evans]

On Fri, 24 Nov 2017, Konstantin Belousov wrote:


On Fri, Nov 24, 2017 at 08:15:06PM +1100, Bruce Evans wrote:

On Thu, 23 Nov 2017, Konstantin Belousov wrote:

* ...

#define pgtok(p)((uintmax_t)(p) * pageKilo)

Amusingly there is already MD macro in machine/param.h with the same name
and same intent, but as you formulate it, sloppy implementation.  It uses
unsigned long cast on almost all 64bit arches, except powerpc.  For 32bit
arches, the cast is not done, unfortunately.


I already pointed out the system pgtok().

It was almost correct to cast to u_int on all arches because the
implementation can know the size of all its page counters.  That was int
or u_int, and the expansion factor is small.  In practice, 32-bit systems
never have enough memory to overflow in K (that happens at 4TB), and
64-bit systems that overflow in K are close to overflowing the page
counters (that happens at 16TB with 4K-pages).

The pgtok() is just unusable because the page size can vary.

No, it is unusable only due to the implementation not ensuring the consistent
output type.


Hmm, I couldn't find any arch with even a compile-time variable PAGE_SIZE.
It is currently just unportable in theory to use hard-coded PAGE_SIZE or
macros that use it.

This might be another leftover from vax days.  getpagesize(2) says:

X  The page size is a system page size and may not be the same as the under-
X  lying hardware page size.
X ...
X  The getpagesize() function appeared in 4.2BSD.

In vax days or even Mach days, PAGE_SIZE might have been the underlying page
size and different from the system page size, so getpagesize() was needed to
provided the latter.  This is sort of backwards.  The system page should be
some small size like 512 that divides the hardware page size for all arches.

Dyson's rewrite might have reversed this.  Anyway, it removed most of the
distinctions between hardware and virtual page sizes.  It still has "clicks"
via the btoc() and other macros, but clicks are conflated with pages of
size PAGE_SIZE.  i386 has i386_btop() which I think is for physical pages,
amd64 has amd646_btop() but never uses it.  It would be better to not
pretend to support "clicks".

POSIX has limits {PAGESIZE} and {PAGE_SIZE} (the latter for XSI).  These
are only runtime-invariant like {OPEN_MAX} was before it supported
setrlimit(), so are hard to use.  BSD utilities like the vmstat have
the same problem with getpagesize() unless its API is changed to
guaratee that it returns PAGE_SIZE and that uis not ifdefed.

The output type isn't a problem.  Consistently uintmax_t would be easier to
use, but would be a pessimization for arches that don't need large sizes.


Perhaps it is a design error to allow the page size to vary or be
anything except 512 in APIs, just like for disk sector sizes.  Most
disk APIs uses units of bytes or 512-blocks.  The physical size may
be different.  Applications need to know the physical memory page size
even less than they need to know the physical sector size.

Not everybody share the warm memories about VAX.  I think there were no
single significant architecture with hardware support for virtual memory,
after the VAX, which used less than 4K sized pages.


512 still gives good units.  1024 would be even better.


Here are my old fixes for this function (to clean it up before removing it):


I picked some of this, mainly I do not want to change the output format.
I am sure that there are scripts around which parse it.


It is hard to parse.

Another bug here is that the vmmeter 'v' values are very easy to parse by
reading them 1 at a time using sysctl -n, but there are no individual
sysctls for the vmtotal 't' values.  Sysctl could usefully expand the
struct as fake integer sysctls (1 per line), but instead prints it
ornately.

I just noticed another bug: sysctl -x is documented to give a raw hex
dump, but for vm.vmtotal it still gives the ornate output.  All sysctls
with special formatting and all strings generated by the kernel seem to
have this bug.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326095 - head/usr.sbin/bsdinstall/scripts

[Bruce Evans]

On Thu, 23 Nov 2017, Devin Teske wrote:


On Nov 23, 2017, at 9:29 AM, Rodney W. Grimes  
wrote:

[ Charset UTF-8 unsupported, converting... ]

On Thu, Nov 23, 2017 at 7:17 AM, Rodney W. Grimes
 wrote:

Also we do provide an ntp.conf so ...


We do, a template, all commented out, and does not work for
machines behind strong firewalls that wont allow ntp out
to the net but have internal ntp servers that are used for
such things.

Well maybe not all commented out, I think it defaults to
some public pools.  I believe it would be missing iburst
for use with ntp -pg


Does ntpdate work out of the box in such environments?  If so, how?


ntpdate time.nist.gov

ntpdate does not need a configureration file, just a command
line argument.




At the banks we used to rely on both (in this order) ntpdate running and then 
ntpd running.


I still rely on this (but I use an old version of ntpd where the removal of
ntpdate is not so advanced).

I think the ntpd developers wanted to remove ntpdate long ago, and thought
that they did not so long ago, but apparently it they didn't because it is
too important for this use.  At best, ntpd would have an option that makes
it behave exactly the same as ntpdate, so removing it would just be a
spelling error.

ntpdate's man page claims this, but is wrong AFAIK.  It says that the
functionality of ntpdate is now available in ntpd(8) using -q.  However
ntpdate -q is far from having equivalent functionality.  According to both
testing of the old version and its current man page, it does the same slow
syncing as normal ntpd startup, and just exits after this and doesn't
daemonize while doing this.  With the old version, this step takes 35-40
seconds starting with the time already synced to withing a few microseconds
(by a previous invocation of ntpd), while ntpdate syncs (perhaps not so
well) with a server half the world away in about 1 second.


Running ntpdate before ntpd meant that on a [re]boot, ntpdate would jump the box to the 
appropriate time, regardless of how far behind the clock was (think "dead cmos 
battery" on a system left powered-off for a long time).


Think "any hardware without an atomic local clock".  With an atomic clock,
the initial sync might take as little as 35-40 seconds because the local
clock is more accurate than the remote clock, but ntpd doesn't know that
so it takes about the same time as starting with a pre-synched clock.

The same problem happens after suspend/resume.  With PC cmos clocks, the
time granularity is 1 second.  It is possible to have an accuracy of better
than 1 second by reading and writing the clock on seconds boundaries, but
clock drift is usually seconds per day so no one bothers to do this (except
some of my versions read the clock on seconds boundaries; writing is harder).
The result is that after an immediate reboot, the PC clock always appears
to be off by up to 2 seconds from a 1 second error in reading it and a
1 second error in writing it (average only half as much).  For reboots
after a few hours, add drift of a few seconds.

The same happens for suspend/resume, even after finally fixing the i386
case in acpi.  Old pmtimer code tried harder to restore a correct time
on resume, but only tried to account for drift _before_ suspend.  acpi
on i386 was missing the restore until a year or 2 ago.  The restore was
only done for amd64 using an amd64 ifdef to break it for other arches.
This was fixed by removing the ifdef.  FreeBSD-10 still has the ifdef.

Anyway, the situation is not much better with the restore after resume
than without it.  The only good way to restore the time after suspend
is to run ntpdate in the resume script, just like it should be run at
boot time.  ntpd -q of course doesn't work, since it would take too
long to resync, or might give up if the difference is too large.

I've never seen a resume script that restores the clock (the example
one in /etc does nothing useful except for 15-20 year old pccard hardware).
Suspsend/resume doesn't work on most of my systems so I didn't write
such a script.  I just removed the -x flag that I was using to prevent
ntpd stepping the time.  All steps were supposed to be done by ntpdate
at boot time or later by the sysadmin, but that doesn't work after
suspend/resume since the only good way to fix up even the small 1-2
second error is to step the clock.  -x was too broken to use for fixing
up large differences.  IIRC, it didn't understand its own corrections
and treated these as further drift, so differences of even 128 seconds
blew out to the abort point of 900 seconds.  After fixing this, small
differences just take too long to fix via slews, and differences of 900
seconds still cause aborts (since that is too large to slew and -x
prevents stepping).  But suspend/resume for just 900+ seconds without the
restore in the kernel always causes such differences.  Reboot behaves
better, even without ntpdate, since it 

Re: svn commit: r326073 - head/usr.bin/systat

[Bruce Evans]

On Thu, 23 Nov 2017, Konstantin Belousov wrote:


On Fri, Nov 24, 2017 at 12:10:09AM +1100, Bruce Evans wrote:

On Thu, 23 Nov 2017, Konstantin Belousov wrote:

* ...

Below is the cast to uintmax_t and unsigned format for sysctl(8).


This adds style bugs by expanding lines from length 79 to 81.


diff --git a/sbin/sysctl/sysctl.c b/sbin/sysctl/sysctl.c
index e1bf4e31914..92685a8171b 100644
--- a/sbin/sysctl/sysctl.c
+++ b/sbin/sysctl/sysctl.c

* ...

All of the casts to uintmax_t can be avoided be avoided by changing the type
of pageKilo from int to uintmax_t.

Better, do the conversion in a function-like macro pgtokb() as is done in
all (?) other utilities, but without so many overflow bugs as in most other
utiities:

#define pgtok(p)((uintmax_t)(p) * pageKilo)

Amusingly there is already MD macro in machine/param.h with the same name
and same intent, but as you formulate it, sloppy implementation.  It uses
unsigned long cast on almost all 64bit arches, except powerpc.  For 32bit
arches, the cast is not done, unfortunately.


I already pointed out the system pgtok().

It was almost correct to cast to u_int on all arches because the
implementation can know the size of all its page counters.  That was int
or u_int, and the expansion factor is small.  In practice, 32-bit systems
never have enough memory to overflow in K (that happens at 4TB), and
64-bit systems that overflow in K are close to overflowing the page
counters (that happens at 16TB with 4K-pages).

The pgtok() is just unusable because the page size can vary.

Perhaps it is a design error to allow the page size to vary or be
anything except 512 in APIs, just like for disk sector sizes.  Most
disk APIs uses units of bytes or 512-blocks.  The physical size may
be different.  Applications need to know the physical memory page size
even less than they need to know the physical sector size.


(pageKilo is back to int).  This is still sloppy:
- pageKilo = getpagesize() / 1024 assumes that the page size is a multiple
   of 1024 and fails very badly when the page size is 512
- the multiplication can still overflow
- when the size in K is too large to fit, the size in M or G will normally
   fit and converting directly to would avoid the overflow assuming that
   the page size is <= 1M.

Using floating point (even float precision) avoids all overflow and rounding
problems up to almost 128-bit sizes in bytes:

No, I do not want to use floating point calculation there.


Why not?  I don't want to use it here, but that is because I don't want
the whole S_vmtotal() function here.  I want to use it in most places that
print large values.  Using it in systat is most natural since floating
point is already used a lot there.  OTOH, I don't like libdevstat using
it, sepecially long double.  It doesn't simplify much except representation
of rates in libdevstat.  The high precision of long double is especially
not needed for device statistics, and the precision cannot be depended on
to be more than double since many arches only have fake long double.  Using
it for libdevstat just pessimizes for arches that have non-fake long double.


diff --git a/sbin/sysctl/sysctl.c b/sbin/sysctl/sysctl.c
index e1bf4e31914..36851f302a0 100644
--- a/sbin/sysctl/sysctl.c
+++ b/sbin/sysctl/sysctl.c
@@ -608,14 +608,18 @@ S_timeval(size_t l2, void *p)
static int
S_vmtotal(size_t l2, void *p)
{
-   struct vmtotal *v = (struct vmtotal *)p;
-   int pageKilo = getpagesize() / 1024;
+   struct vmtotal *v;
+   int pageKilo;

if (l2 != sizeof(*v)) {
warnx("S_vmtotal %zu != %zu", l2, sizeof(*v));
return (1);
}

+   v = p;
+   pageKilo = getpagesize() / 1024;


OK.


+
+#definepg2k(a) ((uintmax_t)(a) * pageKilo)


"2" is a bad abbreviation for "to".  It isn't even shorter, and requires
cetain language skils to understand, and is especially unsuitable for
conversion functions since the digit 2 might mean a conversion factor.



+   printf("Free Memory:\t%juK", pg2k(v->t_free));
+#undef pg2k


No need to undef it.  It is in the application namespace.

Here are my old fixes for this function (to clean it up before removing it):

X Index: sysctl.c
X ===
X RCS file: /home/ncvs/src/sbin/sysctl/sysctl.c,v
X retrieving revision 1.86
X diff -u -2 -r1.86 sysctl.c
X --- sysctl.c  11 Jun 2007 13:02:15 -  1.86
X +++ sysctl.c  25 Sep 2017 07:04:54 -
X @@ -379,5 +381,5 @@
X  {
X   struct vmtotal *v = (struct vmtotal *)p;
X - int pageKilo = getpagesize() / 1024;
X + int pageKilo;
X 
X  	if (l2 != sizeof(*v)) {


I fixed 1 of the initializatons in declarations.  The other one is not
so bad (except for its redundant cast).  This is a common style for void *
args and would be needed with const void * args.

X @@ -385,24 +387,19 @@
X   return (1);
X   }
X -
X - prin

Re: svn commit: r326139 - head/usr.bin/vmstat

[Bruce Evans]

On Thu, 23 Nov 2017, Konstantin Belousov wrote:


Log:
 vmstat: use 64-bit counters from struct vmtotal.

 Consistently print counters using unsigned intmax type.


Not very consistently.  After fixing 0.01% of libxo (just add __printflike()
to xo_emit(), the following errors are detected:

X vmstat.c:817:23: error: format specifies type 'long' but the argument has 
type 'int' [-Werror,-Wformat]
X total.t_rq - 1, total.t_dw + total.t_pw, total.t_sw);
X ^~~
X vmstat.c:817:48: error: format specifies type 'long' but the argument has 
type 'int16_t' (aka 'short') [-Werror,-Wformat]
X total.t_rq - 1, total.t_dw + total.t_pw, total.t_sw);
X  ^~

Broken in r291090 (the first libxo commit to this file).  All the arg types
here are still int since all the fields in 'total' still have type int16_t.

X vmstat.c:865:7: error: format specifies type 'unsigned int' but the argument 
has type 'unsigned long' [-Werror,-Wformat]
X (unsigned long)rate(sum.v_swtch - osum.v_swtch));
X ^~~

Broken in r291090.  Everything in the same printf is broken, but the other
bugs are newer and older:
- in 4.4BSD-Lite2, the the format was %lu for all 3 values to be printed,
  but the type was MD.  It was apparently long.  The struct fields were all
  u_int IIRC, but the rate() expression involves time_t.  time_t was
  _BSD_TIME_T_, which was apparently long on all arches.
- FreeBSD fixed time_t to be int on at least i386.  The type error was
  still non-fatal on 32-bit arches with 32-bit time_t.
- I fixed this in r37453.  The fix wasn't so good.  It retained the %lu
  format and added casts to u_long to match the format.  But the casts are
  unportable and became wrong when v_swtch etc. were expanded.
- r123407 broke the style by changing all u_* to unsigned *.
- the casts became wrong, but fairly harmlessly so, on 32-bit arches when
  v_swtch, etc. was changed to counter_u64_t in r317061 when v_swtch was
  expanded to counter_u64_t.  The problem is limit because the values are
  rays of differences.  Even %u format is probably enough for this.

X vmstat.c:1035:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_swtch);
X ^~~
X vmstat.c:1037:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_intr);
X ^~
X vmstat.c:1039:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_soft);
X ^~
X vmstat.c:1040:38: error: format specifies type 'unsigned int' but the 
argument has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X xo_emit("{:traps/%9u} {N:traps}\n", sum.v_trap);
X  ~~~^~
X  %9lu
X vmstat.c:1042:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_syscall);
X ^
X vmstat.c:1044:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_kthreads);
X ^~
X vmstat.c:1045:46: error: format specifies type 'unsigned int' but the 
argument has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X xo_emit("{:forks/%9u} {N: fork() calls}\n", sum.v_forks);
X  ~~~^~~
X  %9lu
X vmstat.c:1047:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_vforks);
X ^~~~
X vmstat.c:1049:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_rforks);
X ^~~~
X vmstat.c:1051:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_swapin);
X ^~~~
X vmstat.c:1053:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_swappgsin);
X ^~~
X vmstat.c:1055:3: error: format specifies type 'unsigned int' but the argument 
has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
X sum.v_swapout);
X ^
X vmstat.c:1057:3: error: format 

Re: svn commit: r326125 - head/usr.sbin/kgmon

[Bruce Evans]

On Thu, 23 Nov 2017, Edward Tomasz Napierala wrote:


Log:
 Mark kgmon(8) obsolete, redirecting users to pmcstat(8).


It isn't obsolete.  pmcstat can't do at least full (non-statistical)
call graphs and high-resolution profiling.

gmon in the kernel is slow to use in the SMP case and dangerous to use
in all cases, but usually works for call graphs.  In the SMP case, it
uses a giant spinlock which is slow and gives deadlock when a
non-maskable trap like an NMI or debugger trap occurs while the lock
is held by the same CPU.  In the !SMP case, it uses interrupt disabling
to lock.  This races instead of deadlocking for non-maskable traps while
it is held.

Ordinary mutexes have the same problems, and only work if NMI and debugger
trap handlers don't use any mutexes that might be held by the interrupted
context.  Broken cases mostly involve broken locking in printf() and console
drivers.

High-resolution profiling was broken by gcc-4.2.1 and is more broken
for clang.  Only the parts written in asm sort of work, and no parts
ever worked right for SMP.

All these bugs are except the slowness from giant locking are fixed for
gcc-4.2.1 in some of my versions, using better giant locking with a timeout
on it to avoid deadlocking.  When deadlock is detected, profiling is
skipped.  My fixes for printf() and console drivers are similar except
fot trying harder to not skip (switch to alternative methods).

pmc also can't do better than nothing for cases involving NMIs even
when they aren't near deadlock.  Ordinary profiling can do better, and
high resolution profiling can do better still.  E.g., for profiling
an NMI handler, pmc can't generate NMIs to even sample it statistically,
but ordinary profiling can see it whenever the NMI doesn't occur while
the profiling lock is held.  Ordinary profiling then gives an exact
call graph, but broken statistical sampling for times since generating
hardclock interrupts in an NMI (and other contexts) is even more
impossible than generating an NMI for pmc.  High-resolution profiling
gives almost exact times (or perfmon counts) even in NMI handlers
except in the near-deadlock case.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326073 - head/usr.bin/systat

[Bruce Evans]

On Thu, 23 Nov 2017, Konstantin Belousov wrote:


On Thu, Nov 23, 2017 at 04:24:13AM +1100, Bruce Evans wrote:

sysctl/sysctl.c:
   sysctl(8) has bogus support for prettyprinting struct vmtotal (sysctl
   shouldn't have any prettyprinting, especially not for structs that have
   specialized programs to print them and much more).  This uses intmax_t
   for all calculations and printing except for the int16_t fields, so it
   automatically benefited from the expansion.  However since it uses a
   correct type (signed, and not restricted to 64 bits), it now has minor
   type errors -- it dowcasts the uint64_t to intmax_t wheen the latter is
   64 bits.  Its Makefile uses a fairly high WARNS, so if its type errors
   were fatal then printf format checking would have detected them (but
   not non-fatal errors involving downcasting).


Below is the cast to uintmax_t and unsigned format for sysctl(8).


This adds style bugs by expanding lines from length 79 to 81.


diff --git a/sbin/sysctl/sysctl.c b/sbin/sysctl/sysctl.c
index e1bf4e31914..92685a8171b 100644
--- a/sbin/sysctl/sysctl.c
+++ b/sbin/sysctl/sysctl.c
@@ -625,15 +625,15 @@ S_vmtotal(size_t l2, void *p)
"%hd Sleep: %hd)\n",
v->t_rq, v->t_dw, v->t_pw, v->t_sl);
printf(
-   "Virtual Memory:\t\t(Total: %jdK Active: %jdK)\n",
-   (intmax_t)v->t_vm * pageKilo, (intmax_t)v->t_avm * pageKilo);
-   printf("Real Memory:\t\t(Total: %jdK Active: %jdK)\n",
-   (intmax_t)v->t_rm * pageKilo, (intmax_t)v->t_arm * pageKilo);
-   printf("Shared Virtual Memory:\t(Total: %jdK Active: %jdK)\n",
-   (intmax_t)v->t_vmshr * pageKilo, (intmax_t)v->t_avmshr * pageKilo);
-   printf("Shared Real Memory:\t(Total: %jdK Active: %jdK)\n",
-   (intmax_t)v->t_rmshr * pageKilo, (intmax_t)v->t_armshr * pageKilo);
-   printf("Free Memory:\t%jdK", (intmax_t)v->t_free * pageKilo);
+   "Virtual Memory:\t\t(Total: %juK Active: %juK)\n",
+   (uintmax_t)v->t_vm * pageKilo, (uintmax_t)v->t_avm * pageKilo);
+   printf("Real Memory:\t\t(Total: %juK Active: %juK)\n",
+   (uintmax_t)v->t_rm * pageKilo, (uintmax_t)v->t_arm * pageKilo);
+   printf("Shared Virtual Memory:\t(Total: %juK Active: %juK)\n",
+   (uintmax_t)v->t_vmshr * pageKilo, (uintmax_t)v->t_avmshr * 
pageKilo);
+   printf("Shared Real Memory:\t(Total: %juK Active: %juK)\n",
+   (uintmax_t)v->t_rmshr * pageKilo, (uintmax_t)v->t_armshr * 
pageKilo);
+   printf("Free Memory:\t%juK", (uintmax_t)v->t_free * pageKilo);

return (0);
}


All of the casts to uintmax_t can be avoided be avoided by changing the type
of pageKilo from int to uintmax_t.

Better, do the conversion in a function-like macro pgtokb() as is done in
all (?) other utilities, but without so many overflow bugs as in most other
utiities:

#define pgtok(p)((uintmax_t)(p) * pageKilo)

(pageKilo is back to int).  This is still sloppy:
- pageKilo = getpagesize() / 1024 assumes that the page size is a multiple
  of 1024 and fails very badly when the page size is 512
- the multiplication can still overflow
- when the size in K is too large to fit, the size in M or G will normally
  fit and converting directly to would avoid the overflow assuming that
  the page size is <= 1M.

Using floating point (even float precision) avoids all overflow and rounding
problems up to almost 128-bit sizes in bytes:

#define pgtok(p)((p) * pageKilo)/* any integer type * float */
...
float pageKilo;
...
pageKilo = getpagesize() / 1024;

p = INT128_MAX is slightly too large to convert to float.  p =
UINT64_MAX fits in float with reduced precision.  No one cares about
printing more that FLT_DIG = 6 digits for memory sizes, but use double
instead of float if you want to fit INT128_MAX and print up to DBL_DIG
= 15 digits.

Then for printing, change all %jd and %ju to %.0f.  In, systat, remove all
new printing functions and use putfloat.

The problem of printing large values is still there: putfloat might print
4TB bytes as 4398046511k K's or 4398047M K's (where k means 1000, K means
1024 but M means 100 and the K's suffix is implicit, but the new
printing function is little better -- it prints 4294962966K K's or
4194304M K's instead.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326073 - head/usr.bin/systat

[Bruce Evans]

On Wed, 22 Nov 2017, Konstantin Belousov wrote:


On Wed, Nov 22, 2017 at 08:59:21AM +1100, Bruce Evans wrote:

On Tue, 21 Nov 2017, Konstantin Belousov wrote:


Log:
 systat: use and correctly display 64bit counters.

...
Using unsigned types gives sign extension bugs as usual.  In old versions
...

@@ -491,15 +495,15 @@ showkre(void)
putfloat(100.0 * s.v_kmem_map_size / kmem_size,
   STATROW + 1, STATCOL + 22, 2, 0, 1);

-   putint(pgtokb(total.t_arm), MEMROW + 2, MEMCOL + 4, 7);
-   putint(pgtokb(total.t_armshr), MEMROW + 2, MEMCOL + 12, 7);
-   putint(pgtokb(total.t_avm), MEMROW + 2, MEMCOL + 20, 8);
-   putint(pgtokb(total.t_avmshr), MEMROW + 2, MEMCOL + 29, 8);
-   putint(pgtokb(total.t_rm), MEMROW + 3, MEMCOL + 4, 7);
-   putint(pgtokb(total.t_rmshr), MEMROW + 3, MEMCOL + 12, 7);
-   putint(pgtokb(total.t_vm), MEMROW + 3, MEMCOL + 20, 8);
-   putint(pgtokb(total.t_vmshr), MEMROW + 3, MEMCOL + 29, 8);
-   putint(pgtokb(total.t_free), MEMROW + 2, MEMCOL + 38, 7);
+   putuint64(pgtokb(total.t_arm), MEMROW + 2, MEMCOL + 4, 7);
+   putuint64(pgtokb(total.t_armshr), MEMROW + 2, MEMCOL + 12, 7);
+   putuint64(pgtokb(total.t_avm), MEMROW + 2, MEMCOL + 20, 8);
+   putuint64(pgtokb(total.t_avmshr), MEMROW + 2, MEMCOL + 29, 8);
+   putuint64(pgtokb(total.t_rm), MEMROW + 3, MEMCOL + 4, 7);
+   putuint64(pgtokb(total.t_rmshr), MEMROW + 3, MEMCOL + 12, 7);
+   putuint64(pgtokb(total.t_vm), MEMROW + 3, MEMCOL + 20, 8);
+   putuint64(pgtokb(total.t_vmshr), MEMROW + 3, MEMCOL + 29, 8);
+   putuint64(pgtokb(total.t_free), MEMROW + 2, MEMCOL + 38, 7);


I see that a very recent expansion from int32_t to uint64_t didn't work
here.

Can you explain, please ?


Just that systat -v didn't automatically benefit from the expansion (since
it is not very well written, with hard-coded types.  Hard-coded ints were
at least simple.  Now it has lots of hard-coded uint64_t's, partly to
work around dehumanize_number() being broken as designed (this function
doesn't support numbers, but only supports uint64_t.  It would be a bit
much for it to support surreal numbers, but it doesn't even support signed
integers)).

I checked all programs in /usr/src that use t_avm (t_vm is too hard to grep
for).  (There aren't many.  Statistics utilities in ports probably have more
variations and more bugs altogether.)  None automatically benefited from the
change, and all still have sign type errors:
- systat/vmstat.c as above
- vmstat/vmstat.c:
  This is much uglier, since it has flags to control the dehumanized printing
  and uses libxo so it us ugly even without all the lexical style bugs added
  with libxo.

  vmstat does most things worse:
  - with hflag (this gives the dehumanized printing), it tries to print
memory sizes in bytes.  Even humans would have problems reading the
large decimal numbers resulting from sizes in bytes, but limited field
widths usually result in dehumanization to K or M (rarely G or T).
systat -v tries to print sizes in K (without the usually-redundant K
suffix) more consistently, but this has problems when the values in K
don't fit: vmstat does one thing better:
  if the size is too large to express in K, say just 4M, then vmstat
  will print 4M starting with the size in bytes, but vmstat starts with
  the size in K (4096) and will now print this as 4K, while the old
  version printed it as 4k.  4K of a size in K is almost as confusing
  as 4k of a size in K.

This is one of the few places where the expansion worked automatically,
because prthuman() already uses uint64_t (misspelled u_int64_t).  The
casts of sum.v_page_size to u_int64_t is now redundant as well as
misspelled.  It was needed even with int32_t fields because of the
bad units -- 2G in bytes is quite small.  This casts also gave sign
extension/overflow.  Now it has no effect.  Such casts are safer, but
uint64_t hard-coded in so many places the code might as well be
simplified by hard-coding here too.

  - the !hflag case is broken by blind expansion to use more 4 more columns
than are available even when all the fields fit (large numbers
mess this up more).  Also, the headers are broken with or without
hflag (the first header line no longer lines up with the second
header line.

  - without hflag, the code is similar to systat, but much more broken.  It
has the same fairly hard to reach overflows in pgtokb().  Now it has
undefined behaviour in all cases in xo_emit(), since the arg type
changed to uint64_t but the format is still %7d.  Previously this
only had undefined behaviour in overflowing cases (t_avm was int32_t,
but v_page_size is u_int, so pgtokb() had type u_int; in overflow
cases its value exceeds INT_MAX so the behaviour is undefined even
though the overflow didn't give undefined behaviour.

xo_emit() is not declared as __printflike

Re: svn commit: r326073 - head/usr.bin/systat

[Bruce Evans]

On Tue, 21 Nov 2017, Konstantin Belousov wrote:


Log:
 systat: use and correctly display 64bit counters.

 Following struct vmtotal changes, make systat use and correctly
 display 64-bit counters.  Switch to humanize_number(3) to overcome
 homegrown arithmetics limits in pretty printing large numbers.  Use
 1024 as a divisor for memory fields to make it consistent with other
 tools and users expectations.


I don't like dehumanize_number(), and it can't handle most cases in
systat -v since most cases use floating point.

Using unsigned types gives sign extension bugs as usual.  In old versions
version, large unsigned numbers (only their lower 32 bits) were not
unintentionally printed in signed format to get an early warning about
overflow when they reach half-way to 32-bit overflow.  This is no longer
useful, but signed format is still used.  There are related sign extension
bugs for non-64-bit fields that allow the early warning to still work.


Modified: head/usr.bin/systat/vmstat.c
==
--- head/usr.bin/systat/vmstat.cTue Nov 21 19:23:20 2017
(r326072)
+++ head/usr.bin/systat/vmstat.cTue Nov 21 19:55:32 2017
(r326073)
@@ -138,6 +140,8 @@ static float cputime(int);
static void dinfo(int, int, struct statinfo *, struct statinfo *);
static void getinfo(struct Info *);
static void putint(int, int, int, int);
+static void putuint64(uint64_t, int, int, int);
+static void do_putuint64(uint64_t, int, int, int, int);


Style bug (unsorting).


static void putfloat(double, int, int, int, int, int);
static void putlongdouble(long double, int, int, int, int, int);
static int ucount(void);
@@ -491,15 +495,15 @@ showkre(void)
putfloat(100.0 * s.v_kmem_map_size / kmem_size,
   STATROW + 1, STATCOL + 22, 2, 0, 1);

-   putint(pgtokb(total.t_arm), MEMROW + 2, MEMCOL + 4, 7);
-   putint(pgtokb(total.t_armshr), MEMROW + 2, MEMCOL + 12, 7);
-   putint(pgtokb(total.t_avm), MEMROW + 2, MEMCOL + 20, 8);
-   putint(pgtokb(total.t_avmshr), MEMROW + 2, MEMCOL + 29, 8);
-   putint(pgtokb(total.t_rm), MEMROW + 3, MEMCOL + 4, 7);
-   putint(pgtokb(total.t_rmshr), MEMROW + 3, MEMCOL + 12, 7);
-   putint(pgtokb(total.t_vm), MEMROW + 3, MEMCOL + 20, 8);
-   putint(pgtokb(total.t_vmshr), MEMROW + 3, MEMCOL + 29, 8);
-   putint(pgtokb(total.t_free), MEMROW + 2, MEMCOL + 38, 7);
+   putuint64(pgtokb(total.t_arm), MEMROW + 2, MEMCOL + 4, 7);
+   putuint64(pgtokb(total.t_armshr), MEMROW + 2, MEMCOL + 12, 7);
+   putuint64(pgtokb(total.t_avm), MEMROW + 2, MEMCOL + 20, 8);
+   putuint64(pgtokb(total.t_avmshr), MEMROW + 2, MEMCOL + 29, 8);
+   putuint64(pgtokb(total.t_rm), MEMROW + 3, MEMCOL + 4, 7);
+   putuint64(pgtokb(total.t_rmshr), MEMROW + 3, MEMCOL + 12, 7);
+   putuint64(pgtokb(total.t_vm), MEMROW + 3, MEMCOL + 20, 8);
+   putuint64(pgtokb(total.t_vmshr), MEMROW + 3, MEMCOL + 29, 8);
+   putuint64(pgtokb(total.t_free), MEMROW + 2, MEMCOL + 38, 7);


I see that a very recent expansion from int32_t to uint64_t didn't work
here.


putint(total.t_rq - 1, PROCSROW + 2, PROCSCOL, 3);
putint(total.t_pw, PROCSROW + 2, PROCSCOL + 4, 3);
putint(total.t_dw, PROCSROW + 2, PROCSCOL + 8, 3);


This has larger sign extension bugs than before.  All the fields here are
still int16_t.  putint() still takes int args, and so the int16_t's int
converted to int.  The used to be printed as int, but now they are converted
to uint64_t.  They shouldn't be negative, but if they are then the were printed
as negative.  Now the conversion to uint64_t has sign-extension bugs/overflows
for negative values.  Negative values still be printed as negative via further
overflows, but if the values are passed to dehumanize_number(), they are
printed as enormous unsigned values.

Printing everything as 64 bits is a pessimization.  It would probably work
to convert everything to float and use only putfloat().  This gives about
8 digits of accuracy and even that is often too many.  This was not done
mainly because floating point was slower than integers.  Now it might
be faster than uint64_t.

Flots could also be converted to integers except for printing the percentage
and not much more.  A special case for the percentage would be easy to write
and is already partly there to avoid printing 100.0 which is too wide.  This
was not done mainly because 32-bit ints were too small.


@@ -518,13 +522,13 @@ showkre(void)
PUTRATE(v_pdwakeups, VMSTATROW + 9, VMSTATCOL, 8);
PUTRATE(v_pdpages, VMSTATROW + 10, VMSTATCOL, 8);
PUTRATE(v_intrans, VMSTATROW + 11, VMSTATCOL, 8);
-   putint(pgtokb(s.v_wire_count), VMSTATROW + 12, VMSTATCOL, 8);
-   putint(pgtokb(s.v_active_count), VMSTATROW + 13, VMSTATCOL, 8);
-   putint(pgtokb(s.v_inactive_count), VMSTATROW + 14, VMSTATCOL, 8);
-   putint(pgtokb(s.v_laundry_count), VMSTATROW + 15, 

Re: svn commit: r326031 - head/sys/fs/msdosfs

[Bruce Evans]

On Tue, 21 Nov 2017, Bruce Evans wrote:


On Tue, 21 Nov 2017, Bruce Evans wrote:


On Mon, 20 Nov 2017, Conrad Meyer wrote:


Log:
 msdosfs(5): Reflect READONLY attribute in file mode

...
This undoes only a small part of READONLY changes in r254627 (which are
small part of of r253627).  I think r254627 got the READONLY semantics
wrong, but they were almost consistent and well documented in the commit
log.  Now they are more seriously inconsistent.
...

PS: I now see a good reason to not change the READONLY attribute using
permissions (doing so is the inconsistent part of r254627), but perhaps
we should complete the undoing of the changes to ignore READONLY for
permissions.

Under MSDOS or Windows, you have to use attrib to change attributes.
This corresponds to having to use chflags(1) under FreeBSD.  Allowing
chmod(1) to change flags together with permissons gives confusing
non-orthogonal behaviour.  Only old FreeBSD and old(?) Cygwin have to
do that since they don't have UF_READONLY (or even chflags).

Example of non-orthogonal behaviour: the fake default permissions are
-r-xr-xr-x and attributes are R.  You try to change this using chmod
u+w.  This doesn't change the permissions, but it changes the attributes
to 0.  Then on rebooting to msdosfs, the R attribute has been lost.  It
is safer to require using only chflags(1 or 2) to change attributes.


I now see the correct way to fix this.  UF_READONLY should be independent
of the permissions and act like a partial immutable flag.

Changes for this:
- back out the curent commit
- remove toggling of ATTR_READONLY for setattr of the mode (the neglected
  change in r254627)
- add return (EACCES) in msdosfs_setattr() if (accmode & VMUMBLE) &&
  (dep->de_Attributes & ATTR_READONLY).  Clone 3 lines in ffs for this.
  ffs used to use VWRITE for VMUMBLE in ufs_vaccessx(), but now uses
  (VMODIFY_PERMS & ~VADMIN_PERMS) in ufs_vaccess().  I think VWRITE is
  still enough for msdosfs.  msdosfs doesn't use V*PERMS anywhere else.
  ffs also uses VMODIFY_PERMS for the v_type checks.  I think they are
  for ACLs, but msdosfs doesn't support ACLs.

Also check that ATTR_READONLY only prevents writing in MSDOS.  It could
reasonably make more things immutable.  In Windows 7, for readonly files,
the DEL commands gives "Access is Denied", but rm in Cygwin gives the
POSIX behaviour of prompting for removal.

We have lots of experience with the permissions in the mode having little
to do with the actual permissions.  Write bits in them remain set when
the file becomes unwritable when immutable flag(s) or certain other flags
are set.  ACLs also affect accessibilty but not the mode.  They just can't
change the mode since their effect depends on user ids.  The effect of
file flags might also depend on user ids.  UF_IMMUTABLE happens to deny
writability for all users, but it doesn't deny changing itself for root.

Note that after the above changes, ATTR_READONLY denies write access even
for root, while its old effect on the permissions only denied write access
to non-root.  This is probably what is wanted.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326037 - head/share/mk

[Bruce Evans]

On Mon, 20 Nov 2017, Warner Losh wrote:


Author: imp
Date: Mon Nov 20 22:41:22 2017
New Revision: 326037
URL: https://svnweb.freebsd.org/changeset/base/326037

Log:
 Add -mno-avx2 for clang as well as -mno-avx. We don't want either of
 them when asking for no SIMD.


Is this needed.  For SSE, -mno-sse implies -mno-sse[2,3,4,...] so
an multiple set of no-sse* flags are not needed (I have only checked this
for sse2).  Multiple no-sse* flags are also not used.


Modified: head/share/mk/bsd.cpu.mk
==
--- head/share/mk/bsd.cpu.mkMon Nov 20 22:27:33 2017(r326036)
+++ head/share/mk/bsd.cpu.mkMon Nov 20 22:41:22 2017(r326037)
@@ -393,7 +393,7 @@ CFLAGS += ${_CPUCFLAGS}
# (-mfpmath= is not supported)
#
.if ${MACHINE_CPUARCH} == "i386" || ${MACHINE_CPUARCH} == "amd64"
-CFLAGS_NO_SIMD.clang= -mno-avx
+CFLAGS_NO_SIMD.clang= -mno-avx -mno-avx2
CFLAGS_NO_SIMD= -mno-mmx -mno-sse


Nearby example of non-use for SSE.


.endif
CFLAGS_NO_SIMD += ${CFLAGS_NO_SIMD.${COMPILER_TYPE}}


Testing shows that -mavx2 -mno-avx kills all AVX for clang in an example
that only generates AVX instructions (-mavx2 enabls these as a subset).

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325988 - head/sys/libkern

[Bruce Evans]

On Mon, 20 Nov 2017, Ed Maste wrote:


Hi Bruce, notes on a few of the points you raised (I'll look at others later):

On 19 November 2017 at 04:07, Bruce Evans <b...@optusnet.com.au> wrote:


The vendor version is in libc/string.  It has been ANSIfied, but the
libkern version has large churning to "optimize" it.  It is the libkern
version that should have been optimized, since bcmp is unimportant in
the kernel and in most applications, but applications have a wider range
so a few might benefit from optimizing it.


I presume you meant "it is the _libc_ version that should have been
optimized", but could also read this as a distinction between
optimized and scare-quotes "optimized."


Yes, optimize the libc version first and usually throw away the optimizations
since they are negative or too MD or too hard to maintain.

Also, clean up libc versions before changing libkern versions.


bcmp is actually optimized
in the kernel in support.[sS] for all arches except powerpc and riscv,
so optimizing the kernel MI version of it is especially unimportant.
In libc where MD optimizations are more important, they are also not
done for arm, arm64 and sparc64.


This could make a nice small project for someone to take on.


A nice large project :-).  I once tried it for bcopy and bzero, but gave up 
~15 years ago when the MD variations became too complicated.  The variations

would be much larger now.  Simple "rep movsb" is now almost best, except
it takes too long to start up so even a naive C loop should beat it for
sizes up to quite large values (nearly 1K).  AVX might be best, but is
less portable, and sometimes has overheating problems which are avoided by
throttling it.  The naive C loop might be fastest with a non-naive compiler
that generates AVX for it, but how is the compiler going to know if its
target hardware will throttle AVX when the program is run? ...


- libc version copyright comment not marked for indent protection using
  "/*-" (this has been subverted to have another meaning which I forget).


I don't believe there's another meaning. IIRC imp@ has pointed out
that it means precisely what you've mentioned above.


"/*-" was originally just the normal style for indent protection.  indent
also supports "/**" for this.  It is not easily to find copyrights without
a unique markup, so IIRC the "/*-" form was reserved for copyrights, and
other comments that used this were changed to use "/**".  I prefer the
"/*-" form and don't like it to be restricted to copyright comments.
Actually, I seem to have remembered incorrectly.  In kern/*.c, there are
a few old '/*-' comments (there never were very many), and most '/**'
comments are for doxygen.

style(9) has fairly bad documentation of this.  It only mentions the markup
use of '/*-' for copyright comments.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326031 - head/sys/fs/msdosfs

[Bruce Evans]

On Tue, 21 Nov 2017, Bruce Evans wrote:


On Mon, 20 Nov 2017, Conrad Meyer wrote:


Log:
 msdosfs(5): Reflect READONLY attribute in file mode

...
This undoes only a small part of READONLY changes in r254627 (which are
small part of of r253627).  I think r254627 got the READONLY semantics
wrong, but they were almost consistent and well documented in the commit
log.  Now they are more seriously inconsistent.
...
Most relevant parts of the commit log for r254627:

X msdosfs_denode.c,
X msdosfs_vnops.c:  Add support for getting and setting
X   UF_HIDDEN, UF_SYSTEM and UF_READONLY
X   in MSDOSFS.
X X It supported SF_ARCHIVED, but this has been
X   changed to be UF_ARCHIVE, which has the same
X   semantics as the DOS archive attribute instead
X   of inverse semantics like SF_ARCHIVED.
X X After discussion with Bruce Evans, change
X   several things in the msdosfs behavior:
X X Use UF_READONLY to indicate whether a file
X   is writeable instead of file permissions, but
X   don't actually enforce it.
^

I forget if I asked for not enforcing it or for removing the old enforcement.


PS: I now see a good reason to not change the READONLY attribute using
permissions (doing so is the inconsistent part of r254627), but perhaps
we should complete the undoing of the changes to ignore READONLY for
permissions.

Under MSDOS or Windows, you have to use attrib to change attributes.
This corresponds to having to use chflags(1) under FreeBSD.  Allowing
chmod(1) to change flags together with permissons gives confusing
non-orthogonal behaviour.  Only old FreeBSD and old(?) Cygwin have to
do that since they don't have UF_READONLY (or even chflags).

Example of non-orthogonal behaviour: the fake default permissions are
-r-xr-xr-x and attributes are R.  You try to change this using chmod
u+w.  This doesn't change the permissions, but it changes the attributes
to 0.  Then on rebooting to msdosfs, the R attribute has been lost.  It
is safer to require using only chflags(1 or 2) to change attributes.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326031 - head/sys/fs/msdosfs

[Bruce Evans]

ATTR_READONLY;
X + }
X + if (vap->va_flags & UF_SYSTEM)
X + dep->de_Attributes |= ATTR_SYSTEM;
X + else
X + dep->de_Attributes &= ~ATTR_SYSTEM;
X   dep->de_flag |= DE_MODIFIED;
X   }
X 
X @@ -489,21 +507,24 @@

X   error = VOP_ACCESS(vp, VWRITE, cred, td);
X   } else
X   error = VOP_ACCESS(vp, VADMIN, cred, td);
X - if (vp->v_type != VDIR) {
X - if ((pmp->pm_flags & MSDOSFSMNT_NOWIN95) == 0 &&
X - vap->va_atime.tv_sec != VNOVAL) {
X - dep->de_flag &= ~DE_ACCESS;
X - timespec2fattime(>va_atime, 0,
X - >de_ADate, NULL, NULL);
X - }
X - if (vap->va_mtime.tv_sec != VNOVAL) {
X - dep->de_flag &= ~DE_UPDATE;
X - timespec2fattime(>va_mtime, 0,
X - >de_MDate, >de_MTime, NULL);
X - }
X + if ((pmp->pm_flags & MSDOSFSMNT_NOWIN95) == 0 &&
X + vap->va_atime.tv_sec != VNOVAL) {
X + dep->de_flag &= ~DE_ACCESS;
X + timespec2fattime(>va_atime, 0,
X + >de_ADate, NULL, NULL);
X + }
X + if (vap->va_mtime.tv_sec != VNOVAL) {
X + dep->de_flag &= ~DE_UPDATE;
X + timespec2fattime(>va_mtime, 0,
X + >de_MDate, >de_MTime, NULL);
X + }
X + /*
X +  * We don't set the archive bit when modifying the time of
X +  * a directory to emulate the Windows/DOS behavior.
X +  */
X + if (vp->v_type != VDIR)
X   dep->de_Attributes |= ATTR_ARCHIVE;
X - dep->de_flag |= DE_MODIFIED;
X - }
X + dep->de_flag |= DE_MODIFIED;
X   }
X   /*
X* DOS files only have the ability to have their writability

Just after here is where r254627 negects to remove toggling of ATTR_READONLY
for permissions changes.  The current commit touches this, but only fixes a
style bug:


@@ -502,7 +504,7 @@ msdosfs_setattr(struct vop_setattr_args *ap)
}
if (vp->v_type != VDIR) {
/* We ignore the read and execute bits. */
-   if (vap->va_mode & VWRITE)
+   if (vap->va_mode & S_IWUSR)


Style fix.


dep->de_Attributes &= ~ATTR_READONLY;
else
dep->de_Attributes |= ATTR_READONLY;


ATTR_READONLY was only accessible via file flags APIs except for this.
It is write-only here too.

Most relevant parts of the commit log for r254627:

X msdosfs_denode.c,
X msdosfs_vnops.c:  Add support for getting and setting
X   UF_HIDDEN, UF_SYSTEM and UF_READONLY
X   in MSDOSFS.
X 
X 			It supported SF_ARCHIVED, but this has been

X   changed to be UF_ARCHIVE, which has the same
X   semantics as the DOS archive attribute instead
X   of inverse semantics like SF_ARCHIVED.
X 
X 			After discussion with Bruce Evans, change

X   several things in the msdosfs behavior:
X 
X 			Use UF_READONLY to indicate whether a file

X   is writeable instead of file permissions, but
X   don't actually enforce it.
^

I forget if I asked for not enforcing it or for removing the old enforcement.

X 
X 			Refuse to change attributes on the root

X   directory, because it is special in FAT
X   filesystems, but allow most other attribute
X   changes on directories.
X 
X 			Don't set the archive attribute on a directory

X   when its modification time is updated.
X   Windows and DOS don't set the archive attribute
X   in that scenario, so we are now bug-for-bug
X   compatible.

r254627 adds support for UF_READONLY to many other file systems.  msdosfs
should be compatible with the others if possible.  zfs and ffs are most
interesting.  I don't know what zfs does, but ffs doesn't need UF_READONLY
for permissions so it is write only except for reading it as a flag there,
and msdosfs is compatible with that.  The main point of this is that you
can do cp -p back and forth between file systems that support flags without
destroying _all_ of the flags or always getting warnings or errors for
copying failures.  At least compatible flags can be copied safely.  Flags
that are actually used are unlikely to be compatible and might be unsafe
to copy since they give difference secutity holes on different file systems.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325954 - in head: . share/mk sys/conf usr.sbin/config

[Bruce Evans]

On Sat, 18 Nov 2017, Warner Losh wrote:


On Sat, Nov 18, 2017 at 3:42 AM, Bruce Evans <b...@optusnet.com.au> wrote:


since it flags all c99 and newer usages as syntax errors. We've been





This shouldn't break use of lint on K or C90 sources.  But types.h is
broken.  It no longer supports K, C90, or most other things that are
supposed to be controlled by the visibility ifdefs in it.


Thanks for this insightful analysis. I've clipped the rest of it, which is
more of the same. It can't support the new C compiler construct, and we've
managed to break the old ones. Plus it gives lots of warnings that aren't


Lint is just the messenger (until its annotations like NOTREACHED are also
removed).  It can support old C compiler constructs like it used to,
including pre-standard ones that new C compilers/checks might not support.


useful and doesn't understand several of the idioms we've started using
heavily in the kernel (such as forward struct declaration). Since many of
these problems are in the system kernel headers, the break all non-trivial
uses of lint.

None of these things are ever going to be fixed. It's quite likely the


We are never going to fix feature test macros like _POSIX_SOURCE?  Then
removing support for them would give much simpler sources.


kernel will never again compile on anything older than a c99 compiler.
While we have a lot of ifdefs to attempt to support older compilers, it's
almost all certainly bit-rotted from lack of use.


The headers and libraries (non-source parts) should not be limited by the
language used to compile the kernel.  Pascal is not supported, but old
variants of C require only a small amount of code relative to new variants,
so should remain supported.

Kernels don't actually compile with a C99 compiler.  They need -std=gnu99
plus lots of underscores to use extensions.  The kernel is misconfigured
with -std=isomumble:1999 -fms-extensions and only compiles because this
doesn't give strict C99 (+ ms extensions but no gnu extensions).  Userland
is configured correctly with -std=gnu99.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325954 - in head: . share/mk sys/conf usr.sbin/config

[Bruce Evans]

On Sat, 18 Nov 2017, Pedro Giffuni wrote:


On 11/18/17 09:15, Stefan Esser wrote:


OTOH, we should probably consider other tools, like sparse:

https://sparse.wiki.kernel.org/index.php/Main_Page
  The license is fine and it plays nice with the compiler.

It builds on -CURRENT, but the Makefile needs some tweaks (it does
not find LLVM or Gtk+-2.0, even though they are present on my system).

I'll work on a port over the weekend ...


Thanks!
For it to be really useful we still would have to add annotations to the 
kernel headers.


Not just kernel headers, but also user headers and applications.

Ugh, it is the annotations that are the main cost of supporting lint
(or any other checker that needs some).

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325988 - head/sys/libkern

[Bruce Evans]

On Sun, 19 Nov 2017, Ed Maste wrote:


Log:
 ANSIfy sys/libkern

 PR:223641
 Submitted by:  o...@j.email.ne.jp
 MFC after: 1 week


This churns libkern further away from its "vendor" version which is mostly
in libc.  I'm still waiting for you to back out previous churning of
kern/md4.c involving "ANSI"fication.


Modified: head/sys/libkern/ashrdi3.c


The vendor version is in libc/quad.  It hasn't been ANSIfied.  It ws
identical with the libkern version except for changing the include path
to quad.h and a more gratuitous difference to remove sccsid.  The last 2
differences are in most files in libkern and this will not be described
again.


Modified: head/sys/libkern/bcmp.c


The vendor version is in libc/string.  It has been ANSIfied, but the
libkern version has large churning to "optimize" it.  It is the libkern
version that should have been optimized, since bcmp is unimportant in
the kernel and in most applications, but applications have a wider range
so a few might benefit from optimizing it.  bcmp is actually optimized
in the kernel in support.[sS] for all arches except powerpc and riscv,
so optimizing the kernel MI version of it is especially unimportant.
In libc where MD optimizations are more important, they are also not
done for arm, arm64 and sparc64.

libc bcmp uses simple bytewise comparison.  libkern bcmp was optimized
in 1999.  The optimizations are only for little endian, so they only
apply to 3/4 of the arches that don't have an MD version (to riscv
but not the big endian part of powerpc).  The optimizations are to
compare longwords, and this requires complications for endianness.


Modified: head/sys/libkern/bsearch.c


The vendor version is in libc/stdlib.  It has been ANSIfied.  It has an
extra entry point for bsearch_b() and some macros for this.  It has the
following gratuitous differences:
- libc version copyright comment not marked for indent protection using
  "/*-" (this has been subverted to have another meaning which I forget).
- correct non-difference: libc rccsid not removed.  It is ifdefed with
  LIBC_SCCSID so it has no effect in the kernel, so removing it in other
  places in libkern was churning.
- missing blank line near ids in both and differences in #include's confuse
  diff -u into showing identical lines as changed.  libc/stdlib sources
  mostly have the style bug of not separating the sccsid from _FBSDID().
- COMPAR() macro only used in libc.  It is needed for bsearch_b() there,
  but is only used once so the macro for it is just an obfuscation.
  heapsort.c uses the same macro bt defines its own version with many
  more style bugs and uses it 4 times so it is needed there.
- COMPAR() only has the style bug of not having an explicit '*' before
  its function pointer dereference in libc
- only the libkern version has not so proper const poisoning for the
  assignment to 'base'.  At least it doesn't use __DECONST().
- only the libkern version has proper const poisoning for the assignment
  to 'base'
The kernel needs const poisoning more since it is compiled at higher
warning levels, but -Wcast-qual is too strict in the kernel too so is
usually (?) not enabled and/or is broken in clang.


Modified: head/sys/libkern/cmpdi2.c
...
Modified: head/sys/libkern/divdi3.c
...
Modified: head/sys/libkern/lshrdi3.c

Like ashrdi3.c (not ANSIfied in libc/quad).


Modified: head/sys/libkern/mcount.c
==
--- head/sys/libkern/mcount.c   Sat Nov 18 21:39:54 2017(r325987)
+++ head/sys/libkern/mcount.c   Sun Nov 19 00:31:13 2017(r325988)
@@ -56,8 +56,7 @@ __FBSDID("$FreeBSD$");
 * both frompcindex and frompc.  Any reasonable, modern compiler will
 * perform this optimization.
 */
-_MCOUNT_DECL(frompc, selfpc)   /* _mcount; may be static, inline, etc */
-   uintfptr_t frompc, selfpc;
+_MCOUNT_DECL(uintfptr_t frompc, uintfptr_t selfpc) /* _mcount; may be 
static, inline, etc */


The vendor version is in libc/gmon.

The above declaration was already correctly ANSIfied in libc/gmon.  This
unimproves the style by keeping the comment misplaced at the right of the
code where it is a larger style bug than before -- not the line is too long.
ANSIfication in libc/gmon put it on a separate line.


@@ -245,8 +244,7 @@ MCOUNT

#ifdef GUPROF
void
-mexitcount(selfpc)
-   uintfptr_t selfpc;
+mexitcount(uintfptr_t selfpc)
{
struct gmonparam *p;
uintfptr_t selfpcdiff;


This isn't ANSIfied in libc/gmon.

The file still isn't ANSIfied in libkern -- it still has many old-old-style
function definitions near the end for functions taking no args.

The libkern version has nontrivial extensions for the kernel, and a couple
of gratuitous differences in common code.  I use a merged version in some
trees.  libc should support high resolution profiling too, but still doesn't
even support wide counters for ordinary profiling (its 16-bit counters were
not wide enough 30 years ago.  

Re: svn commit: r325954 - in head: . share/mk sys/conf usr.sbin/config

[Bruce Evans]

On Fri, 17 Nov 2017, Warner Losh wrote:


On Fri, Nov 17, 2017 at 6:34 PM, Rodney W. Grimes <
free...@pdx.rh.cn85.dnsmgr.net> wrote:


[ Charset UTF-8 unsupported, converting... ]

Kib@ posted to arch that we were removing it, nobody objected, we

removed

it. If it was a working feature that might have a few users, that's one
thing. But I don't think make lint has actually worked in at least a

decade.

Thats a sad state of affairs.


Indeed.


The bugs are mostly in system sources outside of lint, and user errors.


When I try it today, I get dozens of warnings, and several syntax errors:


types.h(289): syntax error [249]


This only happens with _KERNEL defined.  It shows a bug in types.h.  Debugged
below.


nvme.h(79): syntax error [249]
nvme.h(105): syntax error [249]
nvme.h(137): syntax error [249]
nvme.h(160): syntax error [249]
nvme.h(160): cannot recover from previous errors [224]


nvme.h could easily have lots more bugs than sys/types.h, since it is not
designed to support C90 or applications or be very portable.


since it flags all c99 and newer usages as syntax errors. We've been


This shouldn't break use of lint on K or C90 sources.  But types.h is
broken.  It no longer supports K, C90, or most other things that are
upposed to be controlled by the visibility ifdefs in it.


using

c99 for the kernel since about 8.x, and in the base longer than that.

xlint

hasn't been useful since we started doing this. The types.h change went

in:

c217b5c12e71 (   mdf 2011-12-12 18:44:17 + 289)typedef _Bool

bool;

which as far as I can make out was between 8.x being branched and 9.x

being

branched. This feature has therefore been broken for 3 major releases
without any announcement, or even any bug reports coming in. Not exactly

a

feature that has an audience that needs to be informed of anything...


It was sys/types.h that was broken then.  lint just detects the bug.

The bug in sys/types.h is that iff _KERNEL is defined, it uses _Bool without
declaring it unless
__STDC_VERSION__ < 199901L && __GNUC__ < 3 && !defined(__INTEL_COMPILER__).
stdbool.h uses the same ifdef for defining _Bool.  However, stdbool.h
uses a different definition of bool that hides the bug unless bool is
actually used -- sttdef.h #defines bool as _Bool, while sys/types.h
typedefs bool as _Bool.

The ifdef is fragile and works and fails as follows:
- when __STDC_VERSION__ >= 199901L, _Bool must be defined by the compiler
  and there is no problem.
- gcc-4.2.1 defaults to c90 so __STDC_VERSION__ is undefined and is equivalent
  to 0 (modulo the bug of the existence of -Wundef).  But __GNUC__ is 4, so
  the ifdef is not satisified anyway.  It is important to not typedef _Bool
  since it is a builtin in gcc-4 even with c90.  The __GNUC__ < 3 test is what
  implements this.
- icc apparently defines __GNUC__ as < 3, but defines _Bool, so it is important
  to not typedef _Bool for icc too.  The !defined(__INTEL_COMPILER__) test is
  what implements this.  Most __INTEL_COMPILER__ tests are used because icc
  doesn't emulate being gcc very well.
- clang is very incompatible and defaults to c11, so the ifdef is not
  satisfied by default.  When c90 is forced, clang is less incompatible and
  the ifdef works like for gcc-4 (clang sets __GNUC__ to 4 despite its
  incompatibility, and the incompatibility is not large enough to break this
  test.)

Then in lint, there is a problem getting __STDC_VERSION__ and even __GNUC__
defined correctly.  __GNUC__ should not be defined without the lint option
-g.  lint uses cc -E... instead of cpp to reduce unportabilities, but this
turns out to be fairly unportable too, due to the clang incompatiblity.
The main bugs are that lint doesn't use -std at all, and clang is incompatible.
lint only supports c90 and c90 with some gcc extensions, so should use
-std=c89 or -std=gnu90 to restrict the headers to what it supports.  The
default -std=c89 still works for gcc, but clang gives -std=c11.

lint's main CFLAGS passed to cc are:
-E -x c  (OK so far)
#if 0
bogus null -D__attribute__(()) and -D__extension__(())
#else
-U__GNUC__  (perhaps even worse than the null definitions)
#endif
-Wp,-C -Wcomment (perhaps OK)
-D__LINT__ -Dlint (OK)
-p adds:
-Wtraditional -Wnosystem-headers
-s adds:
-trigraphs -Wtrigraphs -pedantic -D__STRICT_ANSI__
-t adds:
-traditional -D${MACHINE} -D${MACHINE_ARCH}
-d adds:
-nostdinc -idirafter


Something larger.  Just because the src tree use of lint is failing
does not meant that someone out there is not using lint in some
other capacity.  The depreication policy is there to notify them
that lint well be removed as a program in the next release.


Without a working sys/types.h, nothing  non-trivial will work.


Even C90 support is broken.


Lint is totally totally broken. It's a rotting corpse that needs to be just
gone. Since it has been totally broken for several major releases, I see no


Less broken than in 1993 for the 1990 

Re: svn commit: r325825 - head/sys/kern

[Bruce Evans]

On Tue, 14 Nov 2017, Ed Maste wrote:


Log:
 disallow clock_settime too far in the future to avoid panic

 clock_ts_to_ct has a KASSERT that the converted year fits into four
 digits.  By default (sysctl debug.allow_insane_settime is 0) the kernel
 disallows a time too far in the future, using a value of  366-day
 years.  However, clock_settime is epoch-relative and the assertion will
 fail with a tv_sec corresponding to some 8030 years.

 Avoid trying to be too clever, and just use a limit of 8000 365-day
 years past the epoch.

 Submitted by:  Heqing Yan 
 Reported by:   Syzkaller (https://github.com/google/syzkaller)
 MFC after: 1 week
 Sponsored by:  The FreeBSD Foundation


I reported this panic and several others in reply to previous attempted fixes.


Modified: head/sys/kern/kern_time.c
==
--- head/sys/kern/kern_time.c   Tue Nov 14 18:17:23 2017(r325824)
+++ head/sys/kern/kern_time.c   Tue Nov 14 18:18:18 2017(r325825)
@@ -408,7 +408,7 @@ kern_clock_settime(struct thread *td, clockid_t clock_
if (ats->tv_nsec < 0 || ats->tv_nsec >= 10 ||
ats->tv_sec < 0)
return (EINVAL);
-   if (!allow_insane_settime && ats->tv_sec > ULL * 366 * 24 * 60 * 60)
+   if (!allow_insane_settime && ats->tv_sec > 8000ULL * 365 * 24 * 60 * 60)
return (EINVAL);
/* XXX Don't convert nsec->usec and back */
TIMESPEC_TO_TIMEVAL(, ats);


Panics still occur when year 10K is reached via:

tv_sec = 8000ULL * 365 * 24 * 60 * 60 - 1 (approx. 7994.5 years)
POSIX Epoch offset = 1970 years
utc_offset() <= (approx. 7994.5 + 1970) - 10 = approx. -36.5 years
(utc_offset() in seconds is the combined timezone offset
  60 * tz.tz_minuteswest + (wall_cmos_clock ? adjkerntz : 0).)

A utc_offset() of -36.5 years is preposterous but easy to reach without
overflow (utc_offset() has blind overflow at approx. +-69 years).

For 32-bit time_t, both of the above abominable ULL checks are vacuously
false, but panics still occurs for all widths of time_t with non-
preposterous values near the Epoch:

tv_sec = 0  (Epoch in UTC)
utc_offset() = 10 * 60  (garbage or panic in RTC; should be 1969)

Wthout INVARIANTS, this writes the RTC with the garbage time 70/01/01
00:5c:05, where at least the 5c is from a buffer overrun.  With
INVARIANTS, garbage produced by clock_tv_to_ct() is detected there.

   tv_sec = INT_MAX (2038 in UTC)
   utc_offset() = -1(garbage or panic in RTC; should be EINVAL)

This takes 32-bit time_t.  Then tv_sec - utc_offset() blindly overflows
to INT_MIN on supported arches.  The resulting garbage or panic in the
RTC is not much worse than for any other negative value of
tv_sec - utc_offset().

The settime() level doesn't know about utc_offset() (except for the
historical mistake of setting the timezone using settimeofday()), and
this is correct.  Only the RTC level uses utc_offset() (or the timezone).
The correct error handling is to not KASSERT() anything, but avoid overflow
and just don't continue after overflow or set the RTC to a preposterous
or unportable value.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325765 - head/lib/libc/string

[Bruce Evans]

On Mon, 13 Nov 2017, Warner Losh wrote:


 Add notes about overlapping copies.

 Add notes to each of these that specifically state that results are
 undefined if the strings overlap. In the case of memcpy, we document
 the overlapping behavior on FreeBSD (pre-existing). For str*, it is
 left unspecified, however, since the default (and x86) implementations
 do not handle overlapping strings properly.

 PR: 223653
 Sponsored by: Netflix

Modified:
 head/lib/libc/string/memcpy.3
 head/lib/libc/string/strcat.3
 head/lib/libc/string/strcpy.3


C standards and POSIX have much better wording.


Modified: head/lib/libc/string/memcpy.3
==
--- head/lib/libc/string/memcpy.3   Mon Nov 13 16:53:36 2017
(r325764)
+++ head/lib/libc/string/memcpy.3   Mon Nov 13 17:04:44 2017
(r325765)
@@ -54,6 +54,11 @@ bytes from string
.Fa src
to string
.Fa dst .
+If
+.Fa src
+and
+.Fa dst
+overlap, the results are not defined.
.Sh RETURN VALUES
The
.Fn memcpy


The bugs here are:
- src and dst are pointers.  Overlap for pointers makes no sense except
  for pointers to single objects but the pointers here are not for that.
  Standards use the correct description "objects that overlap".
- style bug: verboseness in the nonsense, by naming the non-objects and
  then having to mark up their names.  A full description would be much
  longer, especially for non-counted copies.  For strcpy(), the relevant
  obects are the array of char beginning at .Fa src and ending at its
  terminating NUL, and the array of char beginning at .Fa dst with the
  same number of elements as the source array.  Standards omit all details
  and say simple "objects".  memcpy() only has a count so it is simpler.
  strncpy() has a count in addition to the NUL terminators, so it is more
  complicated.
- use of the wrong term "results".  The correct term is "behavior".  Results
  are are worst implementation-defined in Standards, since it is almost
  impossible for a result to be undefined without the whole behaviour also
  being undefined.  These functions have only 1 result.  In Standard wording,
  function results are return values, not the whole behavior of the function.
- use of the non-technical term "not defined".  Standards use the technical
  term "undefined".

The Standards wording is "If copying takes place between objects that
overlap, the behaviour is undefined".


Modified: head/lib/libc/string/strcat.3
Modified: head/lib/libc/string/strcpy.3

- as above, except "undefined" is spelled correctly

Grepping for "overlap" in draft C99 shows the following other functions
with overlap giving undefined behavior:
   snprintf, sprintf, sscanf, vsnprintf, vsprintf,
   mbstowcs, wcstombs,
   strxfrm, strftime.
   all functions in  where overlap might occur, except when
   explicitly specified (seems to be only explicitly specified for
   wmemmove)

The term "overlap" occurs in 193 section 3 man pages in -current (counting
links as 1 each).  It isi still never used for any of the above functions.
It is used in the following related man pages:
- bcmp.3.  This use is nonsense.  Overlap doesn't cause any problems for
  bcmp().
- bcopy.3 and memmove.3.  This use is not nonsense, but its wording is much
  worse than in Standards.  It says "The .Fn function copies .Fa len bytes
  from string .Fa src to string .Fa dst.  The two strings may overlap.".
  But these functions don't even copy strings, and "may overlap" gives no
  details.  Standards say "The .Fn function copies .Fn n characters from
  the object pointed to be .Fa s2 into the object pointed to by .Fa s1.
  [Complicated 5-line sentence saying that the copying acts as if it is
  done through a temporary object that doesn't overlap.]".  I think standards
  say somewhere, but man pages say nowhere, that copying between
  non-overlapping objects has the defined behaviour of copying the bytes
  (u_char's) it is done by assignment of bytes or a function like memmove()
  (copying of general objects is more complicated since it may skip padding
  or otherwise fail to preserve the representation).  This is needed so that
  the 5-line sentence doesn't need to be much longer.
- memcpy.3.  It was already documented in the BUGS section that memcpy() is
  implemented as bcopy() and therefore the "strings" "may overlap" [then
  portability considerations due to this].  This is buggier than before:
  - old bug: only the MI implementation of memcpy() clearly implements
memcpy() as bcopy().  The amd64, i386 and arm MD implementations
do the same.  mips is backwards for bcopy() and implements it as
memmove().  mips has 2 separate memcpy*.S files, one for plain arm
and one for xscale.  The plain arm one is smaller than memmove.S,
so presumably doesn't support overlapped copies.  The xscale one
is much larger, so has space for overlapped copies and many optimizations
that no longer work.  I don't know what 

Re: svn commit: r325734 - head/sys/amd64/amd64

[Bruce Evans]

On Sun, 12 Nov 2017, Bruce Evans wrote:


On Sun, 12 Nov 2017, Mateusz Guzik wrote:


Log:
 amd64: stop nesting preemption counter in spinlock_enter

 Discussed with:jhb


This seems to break it.  i386 still has the old code.
...
The main broken case is:
- both levels initially 0
- disable interrupts
- raise spinlock count to 1
- bad race window until critical_enter() is called.  Disabling hardware
 interrupts doesn't prevent exceptions like debugger traps or NMIs.
 Debuuger trap handlers shouldn't use critical sections or (spin)
 mutexes, but NMI handlers might.  When an exception handler calls
 spinlock_enter(), this no longer gives a critical section and bad
 things like context switches occur if the handler calls critical_enter/
 exit().
...
I don't like this change.  The nested counting is easier to understand,
and the nested case is very rare and the critical section part of it is
very efficient (then critical_exit() is just lowering the level).  Old
...
I think the nested case is only for recursive spinlocks.  So NMI handlers
should not use any spinlocks and the new bug is small (NMI handlers should
not use non-recursive spinlocks since they would deadlock, and should not
use recursive spinlocks since they don't work).  NMI handlers aren't that
careful.  They call printf(), and even the message buffer has been broken
to use non-recursive spinlocks.


Actually, it is valid for NMI handlers to use spinlocks via mtx_trylock_spin()
in the non-kdb non-panic case, and that is what my fixes for printf() do.

I had confused "nesting preemption counter" (td_critnest) with interrupt
nesting (the bogus td_intr_nesting_level).  td_critnest was incremented
for every concurrently held spinlock, so it could grow quite large without
any interrupt/exception recursion.  So the micro-optimization of td_critnest
is useful if it is faster and fixed to work.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325734 - head/sys/amd64/amd64

[Bruce Evans]

On Sun, 12 Nov 2017, Mateusz Guzik wrote:


Log:
 amd64: stop nesting preemption counter in spinlock_enter

 Discussed with:jhb


This seems to break it.  i386 still has the old code.

It also moves the critical section a little, so that it is inconsistent
for enter/exit.  I think the critical section placement was harmlessly
wrong, and moving it further fixes the new bugs too.


Modified: head/sys/amd64/amd64/machdep.c
==
--- head/sys/amd64/amd64/machdep.c  Sun Nov 12 02:34:33 2017
(r325733)
+++ head/sys/amd64/amd64/machdep.c  Sun Nov 12 03:13:01 2017
(r325734)
@@ -1853,9 +1853,9 @@ spinlock_enter(void)
flags = intr_disable();
td->td_md.md_spinlock_count = 1;
td->td_md.md_saved_flags = flags;
+   critical_enter();
} else
td->td_md.md_spinlock_count++;
-   critical_enter();
}

void


The main broken case is:
- both levels initially 0
- disable interrupts
- raise spinlock count to 1
- bad race window until critical_enter() is called.  Disabling hardware
  interrupts doesn't prevent exceptions like debugger traps or NMIs.
  Debuuger trap handlers shouldn't use critical sections or (spin)
  mutexes, but NMI handlers might.  When an exception handler calls
  spinlock_enter(), this no longer gives a critical section and bad
  things like context switches occur if the handler calls critical_enter/
  exit().

The main old race is:
- as above, except the race is not so bad (I think it is harmless).
  Nested calls to spinlock_enter() can still occur, but at all levels,
  spinlock_enter() never returns without entering a critical section,
  so callers aren't affected.  I think the worst that happens is a
  when nested spinlock_exit() lowers the critical level to 0, context
  switches may occur.  This isn't a large problem since higher spinlock_
  enter() levels are still entering -- they have disabled interrupts,
  but nothing depends on this.

See r214835 for larger bugs related to the old race.  Exceptions cannot
be prevented, and r214835 defends against them only for the spinlock
context.

Defending against preemption is even easier.  Just move the critical_enter()
to before disabling interrupts.


@@ -1865,11 +1865,12 @@ spinlock_exit(void)
register_t flags;

td = curthread;
-   critical_exit();
flags = td->td_md.md_saved_flags;
td->td_md.md_spinlock_count--;
-   if (td->td_md.md_spinlock_count == 0)
+   if (td->td_md.md_spinlock_count == 0) {
+   critical_exit();
intr_restore(flags);
+   }
}

/*


This used to call critical_exit() at the beginning.  This corresponds to
calling critical_enter() at the end of spinlock_enter(), and gives the
same race.

Now it calls critical_exit() after lowering the spinlock count.  This closes
only half of the old race window.

After moving the call to critical_enter() earlier, moving the call to
critical_exit() later to match closes all of the old race window.

The new bug only affects spinlock entry.  For exit, there is just the old
race with a reduced window.

I don't like this change.  The nested counting is easier to understand,
and the nested case is very rare and the critical section part of it is
very efficient (then critical_exit() is just lowering the level).  Old
versions of these functions were written to try to reduce branches and
other bad scheduling for the usual non-nested case.  Perhaps this isn't
done right.  It is intentionally not obfuscated with __predict_*().
spinlock_exit() ended up without the optimization(?) of avoiding the
decrement in the usual case, where spinlock_enter() uses special 0/1
logic for the usual case and needs extra code for the increment in the
unusual case.  Since the slow parts are probably the critical_*() function
calls and the flags changes, perhaps those should be optimized instead.
The old version tried to do this for the function calls by doing them
unconditionally.  This should have worked best for the call in
critical_exit() since the call is first (but should be last).  The order
is critical, and the flags changes use volatile asms which force the order
more than elsewhere.

I think the nested case is only for recursive spinlocks.  So NMI handlers
should not use any spinlocks and the new bug is small (NMI handlers should
not use non-recursive spinlocks since they would deadlock, and should not
use recursive spinlocks since they don't work).  NMI handlers aren't that
careful.  They call printf(), and even the message buffer has been broken
to use non-recursive spinlocks.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325625 - head/sys/sys

[Bruce Evans]

On Fri, 10 Nov 2017, Conrad Meyer wrote:


Log:
 systm.h: Include cdefs.h first

 Ever since r143063, machine/atomic.h requires cdefs.h.  So, include it
 first.  Weak support: style(9) tells us to include cdefs.h first.


This just moves a style bug.  The existing include of sys/cdefs.h was bogus.
machine/atomic.h uses lots of typedefed types, and is not so polluted that
it includes sys/types.h, so it has sys/param.h as a prerequisite (not
sys/types.h since that would be chummy with the implementation -- perhaps
sys/types.h is enough for machine/atomic.h, but sys/systm.h includes the
kitchen sink (but not the kitchen sys/param.h), so it is unclear if
sys/types.h us enough for it.  sys/param.h (and even sys/types.h) includes
sys/cdefs.h, and it is a style bug to not depend on this.

Rather, it is a style bug to not include both sys/param.h and
sys/systm.h in .c files in the kernel, in that order and before any
other includes, or to not depend on the standard pollution of them
including the kitchen and its sink.  In particular, it is a style bug
to include machine/atomic.h directly -- this is part of the sink.
sys/systm.h defines macros like KASSERT() which may be used in any
other header, so all kernel .c files have sys/systm.h as a potential
prerequisite, and the implementation of this is to require explicit
includes of sys/systm.h and not depend on .h files including it as
nonstandard pollution.  Some .h files do have this pollution.  Some
are so bad that they even include sys/param.h too.

r143063 added lots of ifdefs using feature test macros defined in cdefs.h,
and also some ifdefs that sys/cdefs.h is included before it is used.  This
was a good idea at the time, but was badly implemented and is worse now.
The test for sys/cdefs.h being included before machine/atomic.h is now only
done for amd64, i386, mips and powerpc.  The feature tests are rarely done
outside of i386.  Even amd64 mostly doesn't bother with them.  On i386,
they are mostly for gnu asm (especially in i386/include/atomic.h) and for
gcc compiler features that are not in Intel cc.  Gnu asm must now be
considered standard, so tests for it are vacuously true.  Support for Intel
cc has rotted.  clang strictly needs many more feature tests, but it is still
too close to gcc so results of old tests are vacuously the same as for gcc,
and there are no new tests.


 Argument against: since code that includes systm.h still compiles,
 compilation units that include systm.h must already include cdefs.h.  So, an


Yes, they must include it and much more -- see above.  Just using
__FBSDID() requires including sys/cdefs.h before, so the requirement of
including sys/cdefs.h is automatically satisfied in .c files that don't
have the style bug of not using __FBSDID() near the beginning of the file
before other includes.  But that is not enough for sys/systm.h to be self-
sufficient.


 argument could be made that the cdefs.h include could just be removed
 entirely.  That is maybe a bigger change and not one I am interested in
 bikeshedding.


It is just the correct change...


Modified: head/sys/sys/systm.h
==
--- head/sys/sys/systm.hFri Nov 10 01:17:26 2017(r325624)
+++ head/sys/sys/systm.hFri Nov 10 02:00:40 2017(r325625)
@@ -38,10 +38,10 @@
#ifndef _SYS_SYSTM_H_
#define _SYS_SYSTM_H_

+#include 
#include 
#include 
#include 
-#include 
#include 
#include/* for people using printf mainly */


The old include was a style bug in r120610.  sys/queue.h is the last place
that needs an include of sys/cdefs.h before it, since it has always been
documented to be self-sufficient.  In 4.4BSD-Lite2, it doesn't include
sys/cdefs.h.  I don't know if it should have or if FreeBSD grew a dependency
on it.  The dependency was fixed in r99594 by including sys/cdefs.h in
sys/queue.h.  r120610 added a use of __predict_false() in KASSERT() in
sys/systm.h, and added the above old include in a nonsense place.  Adding
it before all other includes wouldn't be nonsense.  It is just a style
bug.

My version of style(9) says:

 Kernel include files (i.e., sys/*.h) come first; normally, 
 or  will be needed before any others.  
 includes .  Do not include both.  Many headers, including
 , include , and it is OK to depend on that.

This doesn't say that it is a style bug to not depend on the other headers
including sys/cdefs.h.  I actually don't like including sys/cdefs at
all.  Even or especially for __FBSDID() -- this alone makes most FreeBSD
sources unportable almost as well as using __P(()) used to..  __FBSDID()
is easy to emulate, but not so if more or subtler macros in it are used.

Kernel sources are inherently unportable, but using sys/cdefs.h in the
implementation is essential and it is a style bug for the implementation
to not understand itself and include sys/cdefs.h in every file.  Only
headers designed to be self-sufficient should 

Re: svn commit: r325386 - head/sys/kern

[Bruce Evans]

On Sun, 5 Nov 2017, Konstantin Belousov wrote:


On Sun, Nov 05, 2017 at 11:42:51AM -0700, Warner Losh wrote:

On Sun, Nov 5, 2017 at 11:32 AM, Conrad Meyer  wrote:


E.g.,

--- a/sys/ufs/ffs/ffs_alloc.c
+++ b/sys/ufs/ffs/ffs_alloc.c
@@ -304,8 +304,7 @@ retry:
}

if (bp->b_blkno == bp->b_lblkno) {
-   if (lbprev >= UFS_NDADDR)
-   panic("ffs_realloccg: lbprev out of range");
+   ASSERT(lbprev < UFS_NDADDR, "ffs_realloccg: lbprev out
of range");
bp->b_blkno = fsbtodb(fs, bprev);
}



Just a side point: All these should be programming errors. The bogus data


This certainly is a programming error.  It has 2 style bugs:
- it obfuscates a simple test and panic.  This is only 4 bytes shorter
- it misformats the new version using a long line.


that comes or could come from the FS itself should remain always-on panics.
Well, actually, they should transition from always-on panics to some sort
of degraded mount that would be more resilient in the face of such
corruption. But failing that, they should remain always-on panics :)


This is what I said in my reply before the last.

I still have no idea what is the point cem tries to express.


As described in other replies, it is to obfuscate panic() using a macro
in cases where the panic is not controlled by INVARIANTS.


Nor I know what should the ASSERT() macro do in the kernel. If the
patch above really about replacing panic() with _K_ASSERT, then I most
likely agree with it, since the error catched is not due to the on-disk
metadata corruption.


KASSERT() can be used in this case.

It is unclear which case applies.  lbprev is the second parameter and
it is unclear if it ever comes directly from the disk or has already
been checked by callers when it did.  The third parameter bprev often
does come directly from the disk.  A typical call is ffs_realloccg(
ip, nb, dp->db[nb], ...).  For this call, undefined behaviour has already
occurred if nb is out of bounds, so KASSERT() is good enough, but if nb
is really insane then the KASSERT() is unreachale since the invalid array
access already paniced.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325386 - head/sys/kern

[Bruce Evans]

On Sun, 5 Nov 2017, Mateusz Guzik wrote:


On Sun, Nov 5, 2017 at 9:15 PM, Konstantin Belousov 
wrote:


On Sun, Nov 05, 2017 at 12:37:50PM -0700, Ian Lepore wrote:

IMO, the only reason ASSERT-style macros exist is to hide the
conditional-on-build-type part of the operation.  That is, to avoid
having #ifdef INVARIANTS scattered everywhere.

bde' point is that KASSERT() is badly designed, and I agree with him.
Now we could at least remove the () around the message formatting part,
but it is too late.


Creating a macro to generate always-on error detection and reporting
code just because there exists a macro to do so conditionally seems to
turn the world on its head.

I agree with this statement.  if()panic(); construct is good enough, IMO.


I agree.


I don't like our panic messages whatsoever, they are quite often not
informative.


I don't like our panic messages, since they are too long.


For instance consider:
if (obj->foo < bar)
   panic("bad foo %d, have fun looking for bar");


I almost prefer panic(".") where the dot is rendered in red :-).  The
dot is too hard to grep for, so I acually prefer panic(
"").  panic("%s", __func__) is even better than
the dot for unreadability.

Often the value of foo and bar and hundreds of other variables printed by
a bloated panic statement are obvious or uninteresting, and you need to
use a debugger and a compiler that does't optimize out or otherwise
obfuscate anything.


Instead a macro akin to PASS(obj->foo, <, bar, "obj %p", obj); can
expand itself to stringify the first 3 terms and also show the compared
values. Saves on boiler-plate written by hand.

I think *all* panics should be accompanied with a linux's oops-like dump.


Dumps can be produced by debuggers or hd on core files.  More than function
names (also offsets within functions) can be produced by debuggers or
addr2line, all without any space/time bloat for stringification.

Some panics already occur via traps.  Then trap() does a simple dump
and panic() can't print anything more useful.  Instead of the red dot
panic, I also prefer a breakpoint or null pointer trap.  Both are
restartble using kdb, unlike panic().  If no backend is attached to kdb,
then these traps turn into panics with a dump from trap().

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325444 - head/sys/kern

[Bruce Evans]

On Sun, 5 Nov 2017, Ed Maste wrote:


Log:
 ANSIfy sys/kern/md4c.c

 PR:223453
 Submitted by:  o...@j.email.ne.jp
 MFC After: 2 weeks


This doesn't ANSIfy md4c.c, but churns its style away from both the
vendor version and the copy in libmd.

The style is very non-KNF, but is preserved to avoid churning.

md4c.c already had prototypes, by declaring extern ones in the headers
(there are further bugs from inconsistent churning in the headers.
 includes , but renames its functions using
excessively ifdefed defines, so its functions don't actually have the
name in their prototypes;  repeats everything in 
except and then makes them inconsistent but still uses macros to
obfuscate the names.  The inconsistencies are:
- libmd MD4Update and MD4Data take a const void *, but kernel MD4Update and
  MD4Data take a const unsigned char *
- libmd MD4Final takes an unsigned char [16], but kernel MD4Final takes an
  unsigned char [__min_size(16)]; old versions hard-coded __min_size() and
  failed to compile with C90 and earlier compilers (also with the
  distribution version of gcc-4.2.1 -std=c99, since it implements n869.txt
  and not C99 in this area); now the kernel version is only the same as the
  libmd version for compilers that don't implement C99.
  C99
- user parts of sys/md4.h are under !_KERNEL .  This is bogus since only
  md4.h should be used if !_KERNEL.  These parts include MD4Final which has
  inconsistent prototypes, so including both headers would fail.
- sys/md4 doesn't declare the FreeBSD extensions MD4Fd, MD4FdChunk and
  MD4FileChunk even under its bogus ifdef.


Modified: head/sys/kern/md4c.c
==
--- head/sys/kern/md4c.cSun Nov  5 19:38:51 2017(r325443)
+++ head/sys/kern/md4c.cSun Nov  5 19:49:44 2017(r325444)


Before this commit, diffs with libmd/md4c.c were only 97 lines, with less
than half the changes for churning and other style bugs.  Some are:
- __FBSDID() is misplaced in the libmd version
- only the libmd version is obfuscated using the CONST_POINTER typedef.
  Both are obfuscated using the POINTER typedef.  (POINTER was to make
  RSA md portable to pre-C90 compilers.  It is harmlessly wrong on C90
  for const pointers, except -Wcast-qual detects the error and -Werror
  makes it harmfully wrong.  The correct cast for C90 with prototypes
  is none, to let the protoype convert.)
- only the libmd version has the good name 'input' for a parameter
  weakened to 'in'.  This might be related to the next renaming.
- only the libmd version has the not so good name 'index' for a parameter
  changed to 'idx'.  The kernel no longer has index(), so the
  name 'index' no longer cause -Wshadow warnings.  -Wshadow has never been
  used for the the kernel, so these warnings didn't occur even when the
  kernel had index().  index() tends to get declared everywhere in libmd
  and the kerenel via namespace pollution in  and ,
  respectively.  Most of the diffs are from renaming 'index'.
- only the libmd version churns bcopy() to memcpy().  This also adds
  bogus casts to POINTER and CONST_POINTER.  These casts are more bogus
  for memcpy where they are used than for bcopy where they are not used,
  since pre-C90 had bcopy() where the casts were needed but might not have
  memcpy() where the casts are not needed for C90 and later.
- similarly to churn bzero() to memset(), except bzero() is better instead
  of different, and the churning also changes the parentheses style from
  KNF to gnu.
- only the kernel version churns and breaks MD4Final().  It "ANSIfies" the
  function, apparently to change [16] to [static 16].  This is not ANSI.
  ANSI C is the earlier American version of C90.  [static 16] is a syntax
  error in C90.  The kernel md4c.c is only used by netsmb.  I don't use
  netsmb, else this syntax error would break compiling with gcc-4.2.1.

Altogther, only about 20 lines of the 97 lines diffs are needed (mainly
14 lines to add weak references).


@@ -90,8 +90,8 @@ static unsigned char PADDING[64] = {

/* MD4 initialization. Begins an MD4 operation, writing a new context.
 */
-void MD4Init (context)
-MD4_CTX *context;/* context */
+void
+MD4Init(MD4_CTX *context)


"ANSIficication" also loses comments on parameter names.  This is an
improvements for banal comments like this, except it is a style bug.
RSA style apparently requires comments.

This also changes from gnu style to KNF for parentheses.  Apparently the
parentheses style change that accompanies churning bzero to memset is a
style fix for some previous change involving bzero.


@@ -107,10 +107,9 @@ MD4_CTX *context;
 operation, processing another message block, and updating the
 context.
 */
-void MD4Update (context, input, inputLen)
-MD4_CTX *context;/* context */
-const unsigned char *input;/* input block 

Re: svn commit: r324853 - in head/sys: kern sys

[Bruce Evans]

On Sun, 5 Nov 2017, Konstantin Belousov wrote:


On Mon, Nov 06, 2017 at 01:46:49AM +1100, Bruce Evans wrote:

On Sun, 5 Nov 2017, Konstantin Belousov wrote:


On Sun, Nov 05, 2017 at 03:06:22AM +1100, Bruce Evans wrote:

On Sun, 22 Oct 2017, Konstantin Belousov wrote:


Author: kib
Date: Sun Oct 22 08:11:45 2017
New Revision: 324853
URL: https://svnweb.freebsd.org/changeset/base/324853

Log:
 Remove the support for mknod(S_IFMT), which created dummy vnodes with
 VBAD type.

 FFS ffs_write() VOP catches such vnodes and panics, other VOPs do not
 check for the type and their behaviour is really undefined.  The
 comment claims that this support was done for 'badsect' to flag bad
 sectors, we do not have such facility in kernel anyway.


badsect(8) is a utility and this commit breaks it by removing its kernel
support.


I see, thank you for the pointer.  I will prepare the patch to remove
badsect(8).


That would increase the number of things to fix.

In which way ?


It requires unremoving badsect(8) as well as its kernel support.


Even if badsect(8) is implemented less hackishly, binary compatibility
for mknod(2) requires ifdefing it with COMPAT_*.

No, mknod(S_IFMT) is clearly non-functional management interface, and we
allow management interface changes between releases, and if desired, inside
the releases as well.


No, it was perfectly functional.  Much more than in 4.1BSD.  Then it was
limited to 16-bit block numbers by dev_t being 16 bits.  With a block size
of 512, that limited it to file systems of size 32 MB.  That wasn't enough
for a small University system in 1981, and Berkeley isn't small.  It was
far too small for FreeBSD-1 in 1993.  32-bit dev_t in 4.4BSD and FreeBSD-2
was large enough until relatively recently.  It is larger than signed 32-bit
daddr_t and ufs1_daddr_t.  Now 64-bit dev_t is larger than signed 64-bit
daddr_t and ufs2_daddr_t.

BTW, I'm now fixing dev_t for newnfs.  I fixed oldnfs to catch up with 32-bit
dev_t in 1996, but newfs for v3 still hard-codes for 16-bit dev_t (actually
for 8-bit major() and minor()), and ABI breakage loses all bits in major()
for v3 and v4 when the kernel uses the new ABI and applications use the old
ABI.  v2 actually works better using raw dev_t's if the server and client
application use the same 32-bit encoding.


I only checked that badsect(*) worked before this commit without INVARIANTS
or soft updates.  Perhaps the panic always occurs with INVARIANTS due to
some INAVRIANTS check being wrong, or soft updates doesn't understand this
case.  badsect(8) says to run badsect on a "quiet" file system, but this
should only be necessary to prevent the bad block being reallocated while
you are creating a VBAD file to hold it.  Note that mknod(S_IFMT) doesn't
force an fsck before the next mount.  This seems to be be just another bug
involving the clean flag -- I doubt that it is really safe to never clean
up the VBAD files, so creating one should have set a sticky dirty flag.

UFS code is not prepared to VBAD vnodes.  I do not mean specifically the
vnode type, but the VOPs operations on such inodes.


But it is prepared when used as documented in badsect(8).


It is pointless to spend the time on making it functional. Modern disk
failure modes made badsect(8) useless in the beginning if the beginning
of 199x.


It is just too hard to use without a more complicated program to control it.

I used a "modern" disk's remapping to fix a few thousand bad sectors which
needed to be fixed or zeroed just to copy as much as possible before
replacing the disk, but the spares ran out too soon.  bad144 would have
worked better.  IIRC, it can handle 64K remaps.  badsect was too hard to
use to even try for a few thousand not even transparent remaps.


I will commit the following patch after make universe finishes.


I don't like this.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324853 - in head/sys: kern sys

[Bruce Evans]

On Sun, 5 Nov 2017, Konstantin Belousov wrote:


On Sun, Nov 05, 2017 at 03:06:22AM +1100, Bruce Evans wrote:

On Sun, 22 Oct 2017, Konstantin Belousov wrote:


Author: kib
Date: Sun Oct 22 08:11:45 2017
New Revision: 324853
URL: https://svnweb.freebsd.org/changeset/base/324853

Log:
 Remove the support for mknod(S_IFMT), which created dummy vnodes with
 VBAD type.

 FFS ffs_write() VOP catches such vnodes and panics, other VOPs do not
 check for the type and their behaviour is really undefined.  The
 comment claims that this support was done for 'badsect' to flag bad
 sectors, we do not have such facility in kernel anyway.


badsect(8) is a utility and this commit breaks it by removing its kernel
support.


I see, thank you for the pointer.  I will prepare the patch to remove
badsect(8).


That would increase the number of things to fix.

Even if badsect(8) is implemented less hackishly, binary compatibility
for mknod(2) requires ifdefing it with COMPAT_*.

I only checked that badsect(*) worked before this commit without INVARIANTS
or soft updates.  Perhaps the panic always occurs with INVARIANTS due to
some INAVRIANTS check being wrong, or soft updates doesn't understand this
case.  badsect(8) says to run badsect on a "quiet" file system, but this
should only be necessary to prevent the bad block being reallocated while
you are creating a VBAD file to hold it.  Note that mknod(S_IFMT) doesn't
force an fsck before the next mount.  This seems to be be just another bug
involving the clean flag -- I doubt that it is really safe to never clean
up the VBAD files, so creating one should have set a sticky dirty flag.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324857 - in head: lib/libc/gen sys/vm

[Bruce Evans]

On Sun, 22 Oct 2017, Edward Tomasz Napierala wrote:


Log:
 Add OID for the vm.overcommit sysctl. This makes it possible to remove
 one call to sysctl(2) from jemalloc startup code. (That also requires
 changes to jemalloc, but I plan to push those to upstream first.)


This change has many style bugs, and the ABI regression is unfortunate.

Is it really useful to avoid 1 syscall, even in critical code?  I prefer
to avoid jemalloc (and also malloc) in small utilities.  In non-small
utilities, 1 more syscall doesn't matter.


Modified: head/lib/libc/gen/sysctl.3
==
--- head/lib/libc/gen/sysctl.3  Sun Oct 22 10:32:40 2017(r324856)
+++ head/lib/libc/gen/sysctl.3  Sun Oct 22 10:35:29 2017(r324857)
@@ -28,7 +28,7 @@
.\"@(#)sysctl.38.4 (Berkeley) 5/9/95
.\" $FreeBSD$
.\"
-.Dd September 10, 2015
+.Dd October 22, 2017
.Dt SYSCTL 3
.Os
.Sh NAME
@@ -741,6 +741,7 @@ privilege may change the value.
.It Dv VM_V_FREE_TARGET Ta integer Ta yes
.It Dv VM_V_INACTIVE_TARGET Ta integer Ta yes
.It Dv VM_V_PAGEOUT_FREE_MIN Ta integer Ta yes
+.It Dv VM_OVERCOMMIT Ta integer Ta yes
.El
.Bl -tag -width 6n
.It Li VM_LOADAVG


This unsorts the list.  Most lists in sysctl(3) were almost sorted, and
this one was unsorted.  The main unsortings and nearby errors now are:
- CTL_VFS is unsorted in the top-level list (and description)
- CTL_P1003_1B is missing in the top-level list (and description)
- the second level for CTL_P1003_1B has more compile-time-constant numeric
  ids than most; these are also missing
- CTL_*MAXID values were bogus/unused.  All of them except the one for
  CTL_P1003_1B have been removed in sys/sysctl.h.
- the second level for CTL_HW is in numeric order which is random/historical
  alphabetically
- the second level KERN_HOSTNAME is unsorted
- the second level KERN_ARND, KERN_DUMPDEV, KERN_IOV_MAX, KERN_IPC,
  KERN_LOGSIGEXIT, KERN_MAXPHYS, KERN_NTP_PLL, KERN_PS_STRINGS and
  KERN_USRSTACK are missing.  All of these are relatively new and probably
  shouldn't exist as compile-time constants.
- the second level KERN_QUANTUM is not missing, but doesn't exist.  Its
  numeric value was renamed to KERN_DUMMY, and its name was changed to
  kern.sched.quantum.  It should never have existed as a compile-time
  constant, but it should be documented.  Its new name is of course not
  documented in any many page.  Nothing in kern.sched is not documented
  in any many page.  sysctl -d kern.sched gives some documentation.
- the second level of KERN_PROF is in numeric/historical order
- the second level PF_ROUTE is unsorted
- the fifth level NET_RT_FLAGS is unsorted and not described
- the fifth level NET_RT_IFLISTL is unsorted
- sysctls without a compile-time-constant numeric id are mostly missing in
  sysctl(3).  The second and lower levels of FP_INET are exceptions which
  show how things should be done.


@@ -773,6 +774,9 @@ process address space when needed.
.It Li VM_V_PAGEOUT_FREE_MIN
If the amount of free and cache memory falls below this value, the
pageout daemon will enter "memory conserving mode" to avoid deadlock.
+.It Li VM_OVERCOMMIT
+Overcommit behaviour, as described in
+.Xr tuning 7 .


This is consistently unsorted.

U.S. spelling for behavior, etc., should be used in most places in FreeBSD
(except where the nearby spelling is already British).

Many references to tuning(7) are bogus, but this one is correct, except
I think the reference should be from tuning(7) to here instead of vice
versa.


.El
.Sh RETURN VALUES
.Rv -std

Modified: head/sys/vm/swap_pager.c
==
--- head/sys/vm/swap_pager.cSun Oct 22 10:32:40 2017(r324856)
+++ head/sys/vm/swap_pager.cSun Oct 22 10:35:29 2017(r324857)
@@ -157,7 +157,7 @@ static vm_ooffset_t swap_reserved;
SYSCTL_QUAD(_vm, OID_AUTO, swap_reserved, CTLFLAG_RD, _reserved, 0,
"Amount of swap storage needed to back all allocated anonymous memory.");
static int overcommit = 0;
-SYSCTL_INT(_vm, OID_AUTO, overcommit, CTLFLAG_RW, , 0,
+SYSCTL_INT(_vm, VM_OVERCOMMIT, overcommit, CTLFLAG_RW, , 0,
"Configure virtual memory overcommit behavior. See tuning(7) "
"for details.");


This reference to tuning(7) is bogus.  The string has several style bugs.
It is verbose, and the reference to the man page is mostly padding.  Every
sysctl should be described in more detail in a man page.  Preferably only
one big one, or one whose name is consistently related to the sysctl name,
so that it is easy to find.


static unsigned long swzone;

Modified: head/sys/vm/vm_param.h
==
--- head/sys/vm/vm_param.h  Sun Oct 22 10:32:40 2017(r324856)
+++ head/sys/vm/vm_param.h  Sun Oct 22 10:35:29 2017(r324857)
@@ -84,7 +84,8 @@
#define VM_V_PAGEOUT_FREE_MIN   9   /* vm_cnt.v_pageout_free_min */

Re: svn commit: r324853 - in head/sys: kern sys

[Bruce Evans]

On Sun, 22 Oct 2017, Konstantin Belousov wrote:


Author: kib
Date: Sun Oct 22 08:11:45 2017
New Revision: 324853
URL: https://svnweb.freebsd.org/changeset/base/324853

Log:
 Remove the support for mknod(S_IFMT), which created dummy vnodes with
 VBAD type.

 FFS ffs_write() VOP catches such vnodes and panics, other VOPs do not
 check for the type and their behaviour is really undefined.  The
 comment claims that this support was done for 'badsect' to flag bad
 sectors, we do not have such facility in kernel anyway.


badsect(8) is a utility and this commit breaks it by removing its kernel
support.

The panic for ffs is a user error (not running fsck after badsect, or not
following the directions for using badsect, or using the privileged
mknod(S_IFMT) operation in an unsafe way).  The bugs for other file systems
are not having a badsect utility and not even disallowing creation of the
vnodes to support badsect.

64-bit dev_t made badsect work better than ever.  It worked as well as
possible (not very well) with at least old badsect and old fsck.  Its
man page has much more bitrot than its code, mostly due to removal of
other utilities:

X BADSECT(8)  FreeBSD System Manager's Manual BADSECT(8)
X 
X NAME

X  badsect - create files to contain bad sectors
X 
X SYNOPSIS

X  badsect bbdir sector ...
X 
X DESCRIPTION

X  The badsect utility makes a file to contain a bad sector.  Normally, bad

Thus has bad grammar and some technical bugs.  "makes a file to contain a bad
sector" should be something like "makes file[s] containing the specified
bad sector[s].  "containing" needs further descripton.  The containers are
fragments which usually consist of multiple sectors.

X  sectors are made inaccessible by the standard formatter, which provides a
X  forwarding table for bad sectors to the driver.  If a driver supports the

Actually, bad sectors are normally made inaccessible by replacing the drive.
The "standard" formatter was removed.

X  bad blocking standard it is much preferable to use that method to isolate
X  bad blocks, since the bad block forwarding makes the pack appear perfect,
X  and such packs can then be copied with dd(1).  The technique used by this

The bad blocking standard was bad144.  It was removed, so using it is
impossible.

X  program is also less general than bad block forwarding, as badsect cannot
X  make amends for bad blocks in the i-list of file systems or in swap
X  areas.

badsect is much more primitive than bad144, but was usable for disks with
just a few bad sectors.

X 
X  On some disks, adding a sector which is suddenly bad to the bad sector

X  table currently requires the running of the standard DEC formatter.  Thus

The standard DEC formatter never existed in FreeBSD.

X  to deal with a newly bad block or on disks where the drivers do not
X  support the bad-blocking standard badsect may be used to good effect.
X 
X  The badsect utility is used on a quiet file system in the following way:


The file system must be quiet else you will get panics before fsck can be
run to fix the file system.

X  First mount the file system, and change to its root directory.  Make a
X  directory BAD there.  Run badsect giving as argument the BAD directory
X  followed by all the bad sectors you wish to add.  (The sector numbers
X  must be relative to the beginning of the file system, but this is not
X  hard as the system reports relative sector numbers in its console error
X  messages.)  Then change back to the root directory, unmount the file

This is now hard.  Non-broken drivers in old versions of FreeBSD used
diskerr(9) to report errors in a uniform way.  This was dumbed down and
renamed disk_err() which is only called by aac, xen/blkfront, virtio/block
and mfi.  This only prints absolute block numbers in unclear units.  GEOM
would have to be involved to print relative sector numbers and partition
names, but I don't know of any GEOM function for this.  I think CAM prints
absolute block numbers (after an error) in a verbose format that is less
useful execpt for SCSI debugging.

I used lm's filelocks or mckusick's prtblknos utility to find block numbers.
Since I don't have any disks with bad sectors handy, I used good sectors.
These utilities print the (relative) ffs fragment number.  This must be
multiplied by fs_frag (default 8) to get the sector number for badsect.

X  system and run fsck(8) on the file system.  The bad sectors should show
X  up in two files or in the bad sector files and the free list.  Have
X  fsck(8) remove files containing the offending bad sectors, but do not
X  have it remove the BAD/n files.  This will leave the bad sectors in
X  only the BAD files.

fsck must be run twice, at least now.

X 
X  The badsect utility works by giving the specified sector numbers in a

X  mknod(2) system call, creating an illegal file whose first 

Re: svn commit: r324457 - in head/sys/dev: pccbb pci

[Bruce Evans]

On Mon, 9 Oct 2017, Alan Somers wrote:


Log:
 Remove embedded newlines from sysctl variable descriptions


This doesn't fix the main bug of writing man pages in sysctl descriptions.


Modified: head/sys/dev/pccbb/pccbb_isa.c
==
--- head/sys/dev/pccbb/pccbb_isa.c  Mon Oct  9 22:19:58 2017
(r324456)
+++ head/sys/dev/pccbb/pccbb_isa.c  Mon Oct  9 22:27:38 2017
(r324457)
@@ -75,11 +75,11 @@ static SYSCTL_NODE(_hw, OID_AUTO, pcic, CTLFLAG_RD, 0,

static int isa_intr_mask = EXCA_INT_MASK_ALLOWED;
SYSCTL_INT(_hw_pcic, OID_AUTO, intr_mask, CTLFLAG_RDTUN, _intr_mask, 0,
-"Mask of allowable interrupts for this laptop.  The default is generally\n\
-correct, but some laptops do not route all the IRQ pins to the bridge to\n\
-save wires.  Sometimes you need a more restrictive mask because some of the\n\
-hardware in your laptop may not have a driver so its IRQ might not be\n\
-allocated.");
+"Mask of allowable interrupts for this laptop.  The default is generally"
+" correct, but some laptops do not route all the IRQ pins to the bridge to"
+" save wires.  Sometimes you need a more restrictive mask because some of"
+" the hardware in your laptop may not have a driver so its IRQ might not 
be"
+" allocated.");


More low quality sysctl messages can be found using

sysctl -da | grep -v ': '  # Sloppy check for embedded newlines
sysctl -da | grep '^[^ ]*: [^A-Z]' # Uncapitalized messages
previous | grep '^dev.*%'' # Auto-generated spam for dev tree
sysctl -da | grep '[^.]*\.$'   # Terminating period
sysctl -da | grep '^[^ ]*: .*\..*\.$' # Multiple sentence, term. period
sysctl -da | grep '^[^ ]*: .*\..*[^.]$' # Period, but not terminating
sysctl -da | grep '\.  '   # Normal sentence break
sysctl -da | grep '\. [^.]'# Misformatted sentence break
sysctl -da | grep '..' # (Really 80 dots).  Long lines

Results on freefall:
- 3294 lines of output
- 7 lines with embedded newlines
- 1041 lines with uncapitalized messages
- 637 lines with previous bug auto-generated for dev tree.  Approx 5 lines
  duplicated for each device
- 109 lines with a terminating period but no other periods
- 5 lines with multiple sentences and a terminating period
- 43 lines with a period but not a terminating period.  This is a sloppy
  check for multiple sentences.  It finds a few correct descriptions with
  periods in sysctl names
- 7 lines with normal sentence breaks.  Half for man pages in sysctls in
  this commit
- 17 lines with single-space sentence breaks
- 359 long lines.  Not counting the sysctl name or ': ', only 30 lines are
  too long.  These lines and ones up to about 10 shorter are also too long
  for source lines.

Another type of search finds inconsistent spellings of booleans.  grep -i
for literal "disable" shows 63 lines with the following inconsistencies
and verboseness:
- often "disable" in the description echoes "disable" in the sysctl name.
  But sometimes the sysctl uses "control", or the sysctl name uses "disable"
  but the description uses "disallow"
- when the description uses "Enable/disable" (10 instances), the name uses
  "allow" (2 instances), "use" (1 instance), "enable "(1 instance) or is
  not clearly boolean (6 instances),
- often (in usb cloned descriptions), "Disable is spelled verbosely as
  "Set to disable".  This is still not verbose enough to say that unset
  means "enable" or even "don't disable".  Disable/enable is never used.

For integer variables, there are lots of inconsistences and verboseness
involving "number/number of/num/n".

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324427 - head/bin/ps

[Bruce Evans]

{Please ignore my previous null reply]

On Mon, 9 Oct 2017, Sergey Kandaurov wrote:


On 9 October 2017 at 13:24, Edward Tomasz Napierala 
wrote:


Author: trasz
Date: Mon Oct  9 10:24:06 2017
New Revision: 324427
URL: https://svnweb.freebsd.org/changeset/base/324427

Log:
  Document "tdnam" keyword to ps(1), and add "tdname" alias.
Modified: head/bin/ps/keyword.c

==
--- head/bin/ps/keyword.c   Mon Oct  9 08:50:03 2017(r324426)
+++ head/bin/ps/keyword.c   Mon Oct  9 10:24:06 2017(r324427)
@@ -201,6 +201,7 @@ static VAR var[] = {
{"tdev", "TDEV", NULL, "terminal-device", 0, tdev, 0, CHAR, NULL,
0},
{"tdnam", "TDNAM", NULL, "terminal-device-name", LJUST, tdnam, 0,
CHAR,
NULL, 0},


Hello,
could you please fix this ^^ "tdnam" description as well?
fwiw, looks like it's only used in libxo as a field name.


Since it is badly designed and implemented, 2 names for it are worse than
for any other keyword.  Perhaps since it was undocumented in the man page
and misdocumented in the screen, nothing uses it and you can rename it
without anything noticing it.

(Design errors start with splitting thread and proc names in a way that
is difficult for applications to untangle.  Implementation errors start
with abusing the space reserved for the old command name (ki_ocomm) for
ki_tdname.  This helps break old applications that only understand
ki_ocomm.  TDNAMLEN = OCOMMLEN = 16 is also too small, and its name is
spelled without an E, unlike all other *NAMELEN names in .
This is too short.  It is only 3 shorter than COMMLEN, but should probably
be longer, depending on how thread and proc names are split.

The last bug has been partly fixed.  There is now ki_moretdname[].  This
has length just 3 (plus 1 for a NUL) to make allow expansion of thread
names to the same length as proc names (19).  ps and top actually
understand this (except the update to use it is missing in ps/ps.c; it
is only in ps/print.c).

Thread names are also printed for the comm and command keywords in ps.
They are essentially appended to ki_comm.  This is too simple of ki_comm
already contains some thread info.  A negative amount of space is
available for expansion.  if the user uses tdnam then it is even more
difficult to get a good combination of proc and thread info.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324427 - head/bin/ps

[Bruce Evans]

On Mon, 9 Oct 2017, Sergey Kandaurov wrote:


On 9 October 2017 at 13:24, Edward Tomasz Napierala 
wrote:


Author: trasz
Date: Mon Oct  9 10:24:06 2017
New Revision: 324427
URL: https://svnweb.freebsd.org/changeset/base/324427

Log:
  Document "tdnam" keyword to ps(1), and add "tdname" alias.

  MFC after:2 weeks
  Sponsored by: DARPA, AFRL

Modified:
  head/bin/ps/keyword.c
  head/bin/ps/ps.1

Modified: head/bin/ps/keyword.c

==
--- head/bin/ps/keyword.c   Mon Oct  9 08:50:03 2017(r324426)
+++ head/bin/ps/keyword.c   Mon Oct  9 10:24:06 2017(r324427)
@@ -201,6 +201,7 @@ static VAR var[] = {
{"tdev", "TDEV", NULL, "terminal-device", 0, tdev, 0, CHAR, NULL,
0},
{"tdnam", "TDNAM", NULL, "terminal-device-name", LJUST, tdnam, 0,
CHAR,
NULL, 0},




Hello,
could you please fix this ^^ "tdnam" description as well?
fwiw, looks like it's only used in libxo as a field name.

[..]

--
wbr,
pluknet


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324178 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua

[Bruce Evans]

On Sun, 1 Oct 2017, Ian Lepore wrote:


On Sun, 2017-10-01 at 22:37 +0200, O. Hartmann wrote:



Author: avg



Log:
\xa0 unbreak kernel builds on sparc64 and powerpc after r324163, ZFS
Channel Programs
\xa0\xa0
\xa0 The custom iscntrl() in ZFS Lua code expects a signed argumnet,
so
\xa0 remove the harmful cast.


[Context lost to mangling of whitespace]

Removing the cast is more fragile than the usual "fix" for warnings
about checking that unsigned values are less than 0.  Robust code must
do such checks in case the type is changed to a correctly signed type,
unless the type is specified as unsigned in an API (then it is the API
that is broken, but this is hard to fix).

Here the behaviour of the function iscntrl(3) is undefined unless the
value is representable as and unsigned char or has the value of the
macro EOF, so iscntl() on a signed character is undefined except when
the value of the character is EOF in which case the result is wrong
iff that character is a control character.  So iscntrl(*s) for char *s
is undefined for about half of all characters when chars are signed.
This is normally "fixed" by casting *s to u_char.  This is broken for
at least the signed char with value EOF if that character is a control
character, and it is unclear what happens for other negative characters,
especially on exotic arches with sizeof(char) == sizeof(int) where
promotion to int doesn't change the size.  The ctype API itself is broken
as designed on these arches (there is no space to keep EOF separate from
characters).

However, the function here is iscntrl(9).  This is only implemented
for cddl and is undodocumented, so no one knows what it does.  Looking
at its internals, it is easy to see that it accepts negative values and
treats them as non-control characters, and that since the check for
negative numbers is the tautologic comparison (C) >= 0 if the arg is
unsigned (as it is on arches with unsigned chars) or just known to
be nonnegative (it might be an unsigned char promoted to int)), there
must be another bug for the warning to not occur in the implementation
too.  This other bug seems to be just that the kernel is not compiled
-Wsystem-headers to unbreak such warnings.  Userland adds -Wsystem-headers
at WARNS >= 1, but kernel makefiles have never added it.  Kernel makefiles
used to use -I- and this should made all headers non-system.  This is
broken now -- -I- was not converted to -Iquote.


Also, removing the cast won't really be a fix on a platform that has
default unsigned chars (which arm does, but arm doesn't use gcc; not
sure about other platforms).


It is really a break on arches with signed char and a different
implementation of iscntrl(), and should have no effect on arches
with unsigned chars, since the arg is alread unsigned then.

gcc has some magic to reduce warnings about tautological comparisons.
Such warnings are usually just harmful, so they shouldn't be printed
unless the compiler can prove that the tautology is independent of
the arch.  The u_char cast allows it to do that.  But when the type is
char, its unsigneness is MD, so the warning should never be printed.
But gcc is apparently not smart enough to do this.  I checked in userland
with -funsigned-char on amd64.  gcc -Wall warns then.  Another bug in
clang is that it doesn't warn then.  clang doesn't even warn about
nnon-forced unsigned char with clang -Wall -Wtautological-compare.
The expression in the test was (c >= 0).  It takes -Wsign-compare to
get a warning about this.

Compilers can prove that values are in a certain range on all arches and
shouldn't warn then either.  The most interesting case is comparing a
loop index against an unsigned variable:

int i;

for (i = 0; i < 1U; i++)
foo();

Compilers shouldn't warn about this, since the int variable is provably
always 0 or 1, and thus cannout e negative, and thus cannot cause any
sign or unsign-extension bugs in the comparison, but ones like clang
-Wsign-compare do.  "Fix"ing this warning by changing i to unsigned
then tends causes actual unsign-extension bugs and further unsigned
poisoning in more complicated code.  The unsigned type for the limit
often occurs naturally as sizeof() due to the bug that sizeof() is
unsigned.  (In K, size_t didn't exist and the type of sizeof()
wasn't specified.  It was presumably int.  This was unimproved by
specifying size_t to be an unsigned type.)

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324007 - head/usr.sbin/mountd

[Bruce Evans]

On Tue, 26 Sep 2017, Emmanuel Vadot wrote:


On Tue, 26 Sep 2017 13:24:57 +0300
Konstantin Belousov  wrote:


On Tue, Sep 26, 2017 at 09:18:18AM +, Emmanuel Vadot wrote:

@@ -1940,14 +1936,16 @@ add_expdir(struct dirlist **dpp, char *cp, int len)
 {
struct dirlist *dp;

-   dp = (struct dirlist *)malloc(sizeof (struct dirlist) + len);
+   dp = (struct dirlist *)malloc(sizeof (struct dirlist));

You might remove the unneeded cast as well.


Right, done in 324012.


Er, mountd has many similar casts, not just the one fixed, and worse ones
like casting dp to caddr_t before passing it to free() (caddr_t is bogus,
and free() doesn't actually take it -- its prototype converts caddr_t to
void *, just like it would convert dp's type to void *)).  Some of these
casts were needed in 1987 for unprototyped pre-StandardC code.  caddr_t
was more needed in 1977 before void * existed (free() takes a char * arg
in K).

But the main hug near here is leaking memory for strdup()).  The above
malloc() allocates 2 storage areas together (1 for the string at the
end), and seems to have a corresponding free().  Now there is an extra
separate storage error for the string and no separate free() for it.

This is not much more than a style bug too.  mountd is a long-lived
daemon, so it should avoid leaking memory, but it would probably work
OK for a few thousand mounts with no free()s at all or a few million
with no frees of strings.  But it tries to free() most things, so it
is a style bug to not try for some.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323516 - in head/sys: dev/bnxt dev/e1000 kern net sys

[Bruce Evans]

On Sat, 16 Sep 2017, Alexander Leidinger wrote:

Quoting Bruce Evans <b...@optusnet.com.au> (from Sat, 16 Sep 2017 13:46:37 
+1000 (EST)):



It gives lesser breakage here:
- with an old PCI em, an error that occur every few makeworlds over nfs now
  hang the hardware.  It used to be recovered from afger about 10 seconds.
  This only happened once.  I then applied my old fix which ignores the
  error better so as to recover from it immediately.  This seems to work as
  before.


As I also have an em device which switches into non-working state: what's the 
patch you have for this? I would like to see if your change also helps my 
device to get back into working shape again.


X Index: em_txrx.c
X ===
X --- em_txrx.c (revision 323636)
X +++ em_txrx.c (working copy)
X @@ -640,9 +640,20 @@
X 
X  		/* Make sure bad packets are discarded */

X   if (errors & E1000_RXD_ERR_FRAME_ERR_MASK) {
X +#if 0
X   adapter->dropped_pkts++;
X - /* XXX fixup if common */
X   return (EBADMSG);
X +#else
X + /*
X +  * XXX the above error handling is worse than none.
X +  * First it it drops 'i' packets before the current
X +  * one and doesn't count them.  Then it returns an
X +  * error.  iflib can't really handle this error.
X +  * It just resets, and this usually drops many more
X +  * packets (without counting them) and much time.
X +  */
X + printf("lem: frame error: ignored\n");
X +#endif
X   }
X 
X  		ri->iri_frags[i].irf_flid = 0;


This is for old em.  nfs doesn't seem to notice the dropped packet(s) after
this.

I think the comment "fixup if common" means "this error should actually
be handled if it occurs enough to matter".

I removed the increment of the dropped packet count because with the change
none are dropped directly here.  I think the error is just for this packet
but more than 1 packet might be dropped by returning in the old code, but
debugging code seem to show no more than 1 packet at a time having an error.
I think returning drops good packets after the bad one together with leaving
the state inconsistent, and it takes almost a reset to recover.

X @@ -703,8 +714,12 @@
X 
X  		/* Make sure bad packets are discarded */

X   if (staterr & E1000_RXDEXT_ERR_FRAME_ERR_MASK) {
X +#if 0
X   adapter->dropped_pkts++;
X   return EBADMSG;
X +#else
X + printf("em: frame error: ignored\n");
X +#endif
X   }
X 
X  		ri->iri_frags[i].irf_flid = 0;


This is for newer em.  I haven't noticed any problems with that (except it
has 27 usec higher latency).

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323516 - in head/sys: dev/bnxt dev/e1000 kern net sys

[Bruce Evans]

On Fri, 15 Sep 2017, Scott Long wrote:


On Sep 13, 2017, at 9:12 AM, Sean Bruno  wrote:


#blamebruno


A large burden.


Reviewed by:sbruno
Approved by:sbruno (mentor)
Sponsored by:Limelight Networks
Differential Revision:https://reviews.freebsd.org/D12235


*gasps at the LoC count and number of changed drivers*

Could someone please better break this up in the future..?


This ridiculous loc was due to the amount of restructuring needed to get
us back to being in sync with our development.  The future will not have
this big of a commit/change and should be more bite/fun sized.

The #blamebruno tag in the commit log was to indicate that this was not
something Stephen was doing on his own, but was something that I had
induced over the last few months during testing.


This breaks my system.  I prior to this commit, I had igb0 and igb1.  Now I
only have igb0, and it\xe2]x80\x99s like igb1 isn\xe2\x80\x99tt even trying to 
attach.  Unfortunately,
this breaks my nfsroot setup, so I need to roll back my tree.  Please let me
know if there\xe2\x80\x99ss anything I can provide to help debugging.


It gives lesser breakage here:
- with an old PCI em, an error that occur every few makeworlds over nfs now
  hang the hardware.  It used to be recovered from afger about 10 seconds.
  This only happened once.  I then applied my old fix which ignores the
  error better so as to recover from it immediately.  This seems to work as
  before.
- with a newer PCI-e em, throughput as measured by netblast is down by
  more that a factor of 2 since last month.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323518 - head/sys/conf

[Bruce Evans]

On Wed, 13 Sep 2017, John Baldwin wrote:


On Wednesday, September 13, 2017 09:09:42 AM Sean Bruno wrote:


On 09/13/17 08:35, John Baldwin wrote:

On Wednesday, September 13, 2017 03:56:04 AM Sean Bruno wrote:

Author: sbruno
Date: Wed Sep 13 03:56:03 2017
New Revision: 323518
URL: https://svnweb.freebsd.org/changeset/base/323518

Log:
  Jenkins i386 LINT build uses NOTES to generate its LINT kernel config.

  ixl(4) isn't in here either, so I'll remove lio(4) too.


ixl missing is a bug, please put it in sys/amd64/conf/NOTES (or better yet,
just fix the build on !amd64)


In the case of lio(4), Cavium is explicitly not compiling for 32 bit
architectures.  I'm inquiring to find out if they want to build on the
other 64bit architectures we have.

In the case of ixl(4), Intel has said that this is not supported on
32bit architectures.

I'm unsure what the best course of action is.


Put it in NOTES for arches that support it.  Strictly it should be in MD
NOTES, but MI notes already has lots of MD buses like pci and drivers
like bge that depend on pci.  Some MD NOTES have many nodevice lines to
kill non-MI things in MI NOTES, but apparently none needs to kill pci
or the hundreds of drivers that depend on it yet.


The bug is more if drivers aren't in NOTES at all.  Any thing listed in
sys/conf/files.* should be in some NOTES file, either sys/conf/NOTES for
things that are MI or sys/${arch}/conf/NOTES for things that are MD.


The most broken drivers aren't even listed in sys/conf/files.*.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323393 - in head/sys: sys vm

[Bruce Evans]

On Mon, 11 Sep 2017, Mateusz Guzik wrote:


On Mon, Sep 11, 2017 at 10:24 AM, Ryan Libby  wrote:


On Mon, Sep 11, 2017 at 12:30 AM, Mateusz Guzik  wrote:
[...]

That said, looking now at the struct I think its use should be retired
from the kernel. It can remain in headers for userspace use.


More about this in another reply.


First, there is a bunch of counter(9) fields. I don't know the original
reasoning. I would expect these counters to be statically defined in a
per-cpu struct.


Changing them to 64 bits was even larger ABI breakage (they could have been
replaced by padding so that they read as 0's but don't break all following
fields).


The actual counter(9) counters are per-CPU.  These are just offsets into
the PCPU area.  See r317061 / D10156 (recent work by glebius).


I mean their ids are allocated on boot, even though these counters are
always present and could be resolved at compilation time. So in order
to update them you have to fetch the id, as opposed to having the code
just do the right thing.

I.e. for stuff which is always there we should just have well known offsets
into per-cpu tables. But I'm not going to try to flame a patch in (or write
one).


vmstat(1) has the necessary support to read the counters from dead kernels.

systat(1) is simpler because it doesn't support dead kernels.  It just
uses the horrible GETSYSCTL() macro/implementation.  vmstat uses the
horrible GET_VM_STATS() macro/implementation instead.  The horribleness
of these includes different bad error handling.  vmstat aborts if a sysctl
no longer exists or has changed it size.  This gives much the same breakage
as an ABI change.  systat only spams the window with an error message.  This
is relatively harmless.  Sometimes the error message even remains visible
so it explains why some fields are displayed as always 0.  The change of
the size for the counters was eventually fixed in both vmstat and systat
by changing the kernel to use the application's size in some cases.

I don't care about dead kernels.  Thus, moving the fields doesn't break
anything for me.  It "only" takes about 1 instructions per field read
to handle the move automatically using sysctl().

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323393 - in head/sys: sys vm

[Bruce Evans]

On Sun, 10 Sep 2017, Mateusz Guzik wrote:


Log:
 Move vmmeter atomic counters into dedicated cache lines

 Prior to the change they were subject to extreme false sharing.
 In particular this change shaves about 3 seconds real time of -j 80 
buildkernel.


Changes that small are hard to measure.  Kernel builds take about 3 seconds
real time altogether (cheating slightly -- this is for a FreeBSD-4 kernel
where the sources are held constant to provide a benchmark and a bloatometer
for newer kernels and userlands which take 10-20 times longer).


Modified: head/sys/sys/vmmeter.h
==
--- head/sys/sys/vmmeter.h  Sun Sep 10 18:08:25 2017(r323392)
+++ head/sys/sys/vmmeter.h  Sun Sep 10 19:00:38 2017(r323393)
@@ -60,6 +60,12 @@ struct vmtotal {
#if defined(_KERNEL) || defined(_WANT_VMMETER)
#include 

+#ifdef _KERNEL
+#define VMMETER_ALIGNED__aligned(CACHE_LINE_SIZE)
+#else
+#define VMMETER_ALIGNED
+#endif
+


Using this breaks the "ABI" which is used by at least vmstat(1).

This has some style bugs:
- verbose macro name.  Using this unimproves the formatting later.
- space instead of tab after #define.  This vfile is vaguely KNF-conformant
  and had only 1 instance of this style bug before (for the idempotency
  ifndef, which has 3 other style bugs:
  - missing blank line before its #endif
  - tab instead of space before the comment on its #endif
  - backwards comment on its #endif).


/*
 * System wide statistics counters.
 * Locking:
@@ -126,14 +132,15 @@ struct vmmeter {
u_int v_free_target;/* (c) pages desired free */
u_int v_free_min;   /* (c) pages desired free */
u_int v_free_count; /* (f) pages free */
-   u_int v_wire_count; /* (a) pages wired down */
-   u_int v_active_count;   /* (q) pages active */
u_int v_inactive_target; /* (c) pages desired inactive */
-   u_int v_inactive_count; /* (q) pages inactive */
-   u_int v_laundry_count;  /* (q) pages eligible for laundering */


Moving these also breaks the "ABI".


u_int v_pageout_free_min;   /* (c) min pages reserved for kernel */
u_int v_interrupt_free_min; /* (c) reserved pages for int code */
u_int v_free_severe;/* (c) severe page depletion point */
+   u_int v_wire_count VMMETER_ALIGNED; /* (a) pages wired down */


The more complicated declaration unimproves the formatting, especially
since the macro name is so long.

Is there a technical reason to move to the end?  style(9) requires
sorting on alignment, with the most aligned fields first, not last
(it says to sort on size, but means alignment).  Is there a reason
to reverse this here.


+   u_int v_active_count VMMETER_ALIGNED; /* (a) pages active */
+   u_int v_inactive_count VMMETER_ALIGNED; /* (a) pages inactive */
+   u_int v_laundry_count VMMETER_ALIGNED; /* (a) pages eligible for
+ laundering */


This does more than move the other fields and unimprove their formatting.
It also changes the lock annotations from (q) to (a).

The formatting is ugliest for the last field.

Naming the macro something like VMMA would things fit better.  It is
stupid to abbreviate VIRTUAL_MEMORY to VM but spell out ALIGNED,
especially when the former is global and the latter is local.  If
the namespace were documented, then the documentation would say that
vm, v_ and VM* are reserved, perhaps without underscores, so it would
not be namespace pollution to use VMMA.

The full undocumented namespace in this file seems to be:
- MAXSLP
- vm* (used without an underscore for struct vmtotal
- t_*
- unknown nested pollution in 
- v_*
- VM_*
But no VM* without an underscore before VMMETER_ALIGNED.

I don't know of any man page where this is undocumented.  The only
man pages that refers to vmmeter.h is vm_set_page_size(9).  The
only other man page that refers to vm_cnt is memguard(9).

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323329 - head/sys/sys

[Bruce Evans]

On Sat, 9 Sep 2017, David Chisnall wrote:


On 8 Sep 2017, at 21:09, Mateusz Guzik  wrote:


Author: mjg
Date: Fri Sep  8 20:09:14 2017
New Revision: 323329
URL: https://svnweb.freebsd.org/changeset/base/323329

Log:
 Allow __builtin_memset instead of bzero for small buffers of known size


This change seems redundant, because modern compilers already do this 
optimisation.  For example:

#include 

char buf[42];

void bz(void)
{
bzero(buf, 42);
}

With clang 4.0 on x86 compiles to:

   pushq   %rbp
   movq%rsp, %rbp
   xorps   %xmm0, %xmm0
   movups  %xmm0, buf+26(%rip)
   movaps  %xmm0, buf+16(%rip)
   movaps  %xmm0, buf(%rip)
   popq%rbp
   retq


This is only a valid optimization for POSIX code.  Compiling with -std=c99
turns it off, even though the non-C99  is included.

Note that -std is broken in CFLAGS for the kernel.  It is c99 (actually
iso9899:1999).  Userland uses the correct standard gnu99.  -std=c99 breaks
other gnu features like unnamed struct/unions which are then unbroken
bogusly using -fms-extensions.


Neither contains a call, both have inlined the zeroing.  This change is 
strictly worse, because the compiler has some carefully tuned heuristics that 
are set per target for when to inline the memset / bzero and when to call the 
function.  These are based on both the size and the alignment, including 
whether the target supports misaligned accesses and whether misaligned accesses 
are cheap.  None of this is captured by this change.


It is strictly better than plain bzero() in the kernel, and makes no
difference in userland.


In the kernel, this optimisation is disabled by -ffreestanding, however 
__builtin_memset will be turned into a memset call if the size is not constant 
or if the memset call would be more efficient (as determined by the 
aforementioned heuristics).  Simply using __builtin_memset in all cases should 
give better code, and is more likely to be forward compatible with future ISAs 
where the arbitrary constant picked in this patch may or may not be optimal.


Compilers don't really understand this.  In the kernel, extern memset()
is strictly worse than extern bzero(), since memset() is just a wrapper
around bzero().  Thus simply using __builtin_memset() gives worse
strictly code in all cases that are not inlined.  Not a large amount
worse since the wrapper only does an extra branch and function call
when it reduces to bzero().  memset() is just slow in other cases.

The constant is not arbitrary, but is hard-coded to the x86 value which is
more arbitrary.  x86 compilers know that they don't really understand memory
so they hard-code a conservative value of 64.  It is only possible do better
by compiling for a specific arch and knowing how good or bad the extern
function is.  64 is about the smallest value above which the extern function
could possibly do significantly better by knowing or doing more.  Usually it
does slightly worse near 64 and also higher since it knows less, especially
if the compilation was for a specific arch (clang seems to have even more
methods than my benchmarks for this -- hundreds instead of tens --, while
x86 kernels have only one method optimized for 25-year old arches and now
OK again on new arches with fast string instructions (above a few hundred
bytes).

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323346 - in head/sys: powerpc/powerpc riscv/riscv

[Bruce Evans]

On Sat, 9 Sep 2017, Mateusz Guzik wrote:


Log:
 Fix riscv and powerpc compilation after r323329.

 On these archs bzero is a C function, which triggers a compilation error
 as the compiler tries to expand the macro.


bzero() is a C function on all arches.  It is just written in C on these
archs.


Modified: head/sys/powerpc/powerpc/machdep.c
==
--- head/sys/powerpc/powerpc/machdep.c  Sat Sep  9 05:50:47 2017
(r323345)
+++ head/sys/powerpc/powerpc/machdep.c  Sat Sep  9 05:56:04 2017
(r323346)
@@ -417,43 +417,6 @@ powerpc_init(vm_offset_t fdt, vm_offset_t toc, vm_offs
(sizeof(struct callframe) - 3*sizeof(register_t))) & ~15UL);
}

-void
-bzero(void *buf, size_t len)
[... 100+ more lines of patches]


Ugh.  All you have to do here is declare bzero correctly:

void
(bzero)(void *buf, size_t len)

Since bzero() is now a macro, the old declaration is incorrect.


...
@@ -555,3 +518,41 @@ DB_SHOW_COMMAND(spr, db_show_spr)
(unsigned long)spr);
}
#endif
+
+#undef bzero
+void
+bzero(void *buf, size_t len)


The undef should work too.  Why make the patch unreadable by moving
everything?


Modified: head/sys/riscv/riscv/machdep.c
==
--- head/sys/riscv/riscv/machdep.c  Sat Sep  9 05:50:47 2017
(r323345)
+++ head/sys/riscv/riscv/machdep.c  Sat Sep  9 05:56:04 2017
(r323346)
@@ -151,16 +151,6 @@ cpu_idle_wakeup(int cpu)
return (0);
}

-void
-bzero(void *buf, size_t len)
-{
-   uint8_t *p;
-
-   p = buf;
-   while(len-- > 0)
-   *p++ = 0;
-}
-


This is an underengineered C version, while the powerpc version is
misoverengineered.  The correct overengineering is to copy the libc
version (at worst the MI version which is a bit like the powerpc
kernel version) to libkern.


int
fill_regs(struct thread *td, struct reg *regs)
{
@@ -890,4 +880,15 @@ initriscv(struct riscv_bootparams *rvbp)
riscv_init_interrupts();

early_boot = 0;
+}
+
+#undef bzero
+void
+bzero(void *buf, size_t len)
+{
+   uint8_t *p;
+
+   p = buf;
+   while(len-- > 0)
+   *p++ = 0;
}


I guess moving it to the end is to keep the macro alive for the rest of
the file.  So just declare it correctly and don't use #undef or move
anything.

My old version of this has related complications:

X Index: systm.h
X ===
X RCS file: /home/ncvs/src/sys/sys/systm.h,v
X retrieving revision 1.208
X diff -u -2 -r1.208 systm.h
X --- systm.h   17 Jun 2004 17:16:52 -  1.208
X +++ systm.h   7 Feb 2007 23:20:06 -
X @@ -140,45 +142,79 @@
X  void backtrace(void);
X  void cpu_boot(int);
X -void cpu_rootconf(void);
X -extern uint32_t crc32_tab[];
X +void cpu_halt(void) __dead2;
X +void cpu_reset(void) __dead2;
X  uint32_t crc32(const void *buf, size_t size);
X +extern uint32_t crc32_tab[];
X  void critical_enter(void);
X  void critical_exit(void);
X +void hexdump(void *ptr, int length, const char *hdr, int flags);
X +#define  HD_COLUMN_MASK  0xff
X +#define  HD_DELIM_MASK   0xff00
X +#define  HD_OMIT_COUNT   (1 << 16)
X +#define  HD_OMIT_HEX (1 << 17)
X +#define  HD_OMIT_CHARS   (1 << 18)
X  void init_param1(void);
X  void init_param2(long physpages);
X  void init_param3(long kmempages);
X  void tablefull(const char *);
X -int  kvprintf(char const *, void (*)(int, void*), void *, int,
X - __va_list) __printflike(1, 0);
X +int  kvprintf(char const *, void (*)(int, void*), void *, int, __va_list)
X + __printflike(1, 0);
X  void log(int, const char *, ...) __printflike(2, 3);
X  void log_console(struct uio *);
X  int  printf(const char *, ...) __printflike(1, 2);
X  int  snprintf(char *, size_t, const char *, ...) __printflike(3, 4);
X -int  sprintf(char *buf, const char *, ...) __printflike(2, 3);
X +int  sprintf(char *, const char *, ...) __printflike(2, 3);
X +int  sscanf(const char *, char const *, ...) __scanflike(2, 3);
X +long strtol(const char *, char **, int) __nonnull(1);
X +quad_t   strtoq(const char *, char **, int) __nonnull(1);
X +u_long   strtoul(const char *, char **, int) __nonnull(1);
X +u_quad_t strtouq(const char *, char **, int) __nonnull(1);
X +void tprintf(struct proc *p, int pri, const char *, ...)
X + __printflike(3, 4);
X  int  uprintf(const char *, ...) __printflike(1, 2);
X  int  vprintf(const char *, __va_list) __printflike(1, 0);
X  int  vsnprintf(char *, size_t, const char *, __va_list) __printflike(3, 0);
X -int	vsnrprintf(char *, size_t, int, const char *, __va_list) __printflike(4, 0); 
X -int	vsprintf(char *buf, const char *, __va_list) __printflike(2, 0);

X +int  vsnrprintf(char *, size_t, int, const char *, __va_list)
X +	__printflike(4, 0); 
X +int	vsprintf(char *, const char *, __va_list) __printflike(2, 0);

X +int  

Re: svn commit: r323192 - head/contrib/binutils/opcodes

[Bruce Evans]

On Tue, 5 Sep 2017, Ryan Libby wrote:


Log:
 gnu binutils: FSGSBASE assembly/disassembly

 Enable the in-tree binutils to assemble and disassemble amd64 FSGSBASE
 instructions (rdfsbase, rdgsbase, wrfsbase, wrgsbase), used in the base
 system since r322763.


Thanks.


 This gives one last gasp for in-tree gcc, and provides a small
 enhancement for in-tree binutils objdump.


After this, it will be necessary to use .byte.  This is almost simpler
for *fsbase too.

ddb still doesn't understand this instruction or many other more
important instructions.  amd64 is most broken, but I rarely notice
since I normally use i386.  ddb still doesn't understand pc-relative
addressing so for amd64 it displays addresses of global variables as
raw offsets from %rip.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323329 - head/sys/sys

[Bruce Evans]

On Fri, 8 Sep 2017, Ngie Cooper (yaneurabeya) wrote:


On Sep 8, 2017, at 13:09, Mateusz Guzik  wrote:
...
Log:
 Allow __builtin_memset instead of bzero for small buffers of known size


This change breaks all platforms that use gcc. If it???s not reverted in the 
next 3 hours, I???ll revert it.


Not all.  It works for me.  My version of it is 15-20 years old and never
always worked on old versions of FreeBSD

The problem seems to be with inlining parameters.  These are fairly broken
by default in gcc-4.2.1, and the overrides in kern/pre.mk only work in some
cases.  -Os is completely broken (with the overrides, it at best gives a
20% pessimization for space, and with other overrides inlining tends to
fail).  I recently found a workaround: compile with -fno-inline-functions.
So my full compiler flags are CONF_CFLAGS=[-march=i386] -Os
-fno-inline-functions -fno-inline-functions-called-once.  This should
reduce problems with inlining.  However, kernels still build with the
bad default flags of -O2.


Please fix ASAP if possible: 
https://ci.freebsd.org/job/FreeBSD-head-riscv64-build/3668/console .


This also shows problems with uninitialized variables.  But
-Wno-uninitialized is the default for gcc-4.2.1.  This breaks lots of
warnings Maybe you have different nonstandard options to avoid the
breakage.

-Wno-uninitialized is set in kern.mk, and also in kern.pre.mk for cddl.
(CDDL_CFLAGS apparerently gives a dumbed down subset of CWARNFLAGS to
break many more warnings.)  There is a problem getting flags passed
to all Makefiles.  CONF_CFLAGS (and all other makeoptions in config
files) is broken for modules (since macros in the kernel Makefile are
not inherited).  This feature prevents me using modules even if I
forget I don't like them.  Flags in kern.mk are only broken for boot
Makefiles which neglect to include kern.mk (that is, for all current
boot Makefiles).

Bruce___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322969 - in head: sbin/mdconfig sys/dev/md sys/sys

[Bruce Evans]

On Tue, 29 Aug 2017, John Baldwin wrote:


On Tuesday, August 29, 2017 12:18:18 PM Maxim Sobolev wrote:

John, OK, maybe you are right and the current status quo was just an
accident. I am curious what do you and other people think about expressing
expected structure size and padding more explicitly instead of trying to
accommodate for sometimes intricate play between alignment and type size
with something like char[N]? I.e. along the following lines:

#if __WORDSIZE < 64


Use #ifdef __LP64__


#define MD_IOCTL_LEN436
#else
#define MD_IOCTL_LEN448
#endif


It is better to not use any ifdefs.  Ones like this guarantee that the
struct depends on __LP64__, so that it will need translation layer to
work with 32-bit binaries.


struct md_ioctl {
union {
struct _md_ioctl_payload {
unsignedversion; /* Structure layout version */
unsignedunit;/* unit number */


Even unsigned is unportable in theory.  It is portable in practice since
all systems are Vaxes, so they have 32-bit ints.


enum md_types   type ;   /* type of disk */


The size of an enum is very unportable, enums should never be used in ABIs.


char*file;   /* pathname of file to mount */


Pointers are very unportable.  uintptr_t is no better, since its size depends
on the arch and is usually the same as sizeof(void *).  ABIs should use some
semi-portable representation like uint64_t.  This might still require a
translation layer to convert it to a pointer.


off_t   mediasize;   /* size of disk in bytes */


off_t is unportable in theory.  It exists so that its size can be changed.
It is portable in practice because systems are superVaxes, so they have
62-bit off_t's.

uintmax_t is only portable among superVaxes too.  This will break sooner
than off_t.  The existence of __uint128_t already breaks uintmax_t if
you use __uintmax_t.  (uintmax_t is supposed to be the largest type,
but it isn't if it is 64 bits and __uint128_t exists.  This can't be
fixed simply by expanding uintmax_t, since Standard libraries are required
to support uintmax_t but have little support for 128-bit integers, and
more seriously the expansion would be a larger pessimization than using
64-bit uintmax_t and would expose the brokenness of ABIs with uintmax_t
in them.)


unsignedsectorsize;  /* sectorsize */
unsignedoptions; /* options */
u_int64_t   base;/* base address */


This is a portable ABI, but hard-codes some limits, but 64 bits should
be enough for anyone.


int fwheads; /* firmware heads */
int fwsectors;   /* firmware sectors */
char*label;  /* label of the device */


A new pointer.


} md;
char raw[MD_IOCTL_LEN]; /* payload + padding for future ideas */
};
};
CTASSERT(sizeof(struct md_ioctl) == MD_IOCTL_LEN);


This is not the style we use in other structures in FreeBSD.  Simply making
the existing MDNPAD depend on the #ifdef would be more consistent.  For a
really good example of how to handle padding, see kinfo_proc which has
separate "spare" arrays for int, long, void *, and char.


Those arrays are bugs.  They guarantee that the ABI depends on the register
size.  KINFO_PROC_SIZE indeed is very different on amd64 and i386.  But the
i386 ps and other statistics utilities using kinfo work on amd64, so there
must be a translation layer.

Newer structs in  are better designed and use mostly integer
fields of type uint64_t, uint32_t and int.  The main kinfo struct has
many historical mistakes.  It starts with 2 ints, then has many dubious
(kernel-only?) pointers, then uses pid_t.  pid_t is unportable like off_t.
Later, many more typedefed integer types.  The worst old mistakes are the
rusage struct types.  The worst new mistake is the priority struct type
(old versions of kinfo had 3 (?) integer priorities of more interest to
applications, while the struct has has kernel internals in an inconvenient
form).

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322969 - in head: sbin/mdconfig sys/dev/md sys/sys

[Bruce Evans]

On Mon, 28 Aug 2017, Ryan Libby wrote:


On Mon, Aug 28, 2017 at 11:24 AM, Maxim Sobolev  wrote:

Hi John,

Thanks for your feedback! To address the points that you've raised:

1. I've tested on both 32 and 64 bit platforms, it seems not to be the
case. See imp's comment and my reply here
https://reviews.freebsd.org/D10457#216855 . Did I miss something? Can you
post piece of C code that produces different sizeof(struct old) vs.
sizeof(struct new) on some platform?

[...]

On Mon, Aug 28, 2017 at 9:19 AM, John Baldwin  wrote:


On Monday, August 28, 2017 03:54:08 PM Maxim Sobolev wrote:

Author: sobomax
Date: Mon Aug 28 15:54:07 2017
New Revision: 322969
URL: https://svnweb.freebsd.org/changeset/base/322969

Log:
  Add ability to label md(4) devices.


This patch has some style bugs (mainly unsorting of options list in
different ways by not always adding to the end of almost-sorted lists).


Modified: head/sys/sys/mdioctl.h


==

--- head/sys/sys/mdioctl.hMon Aug 28 14:49:26 2017(r322968)
+++ head/sys/sys/mdioctl.hMon Aug 28 15:54:07 2017(r322969)
@@ -49,7 +49,7 @@ enum md_types {MD_MALLOC, MD_PRELOAD, MD_VNODE, MD_SWA
  * Ioctl definitions for memory disk pseudo-device.
  */

-#define MDNPAD   97
+#define MDNPAD   96
 struct md_ioctl {
  unsignedmd_version; /* Structure layout version */
  unsignedmd_unit;/* unit number */
@@ -61,6 +61,7 @@ struct md_ioctl {
  u_int64_t   md_base;/* base address */
  int md_fwheads; /* firmware heads */
  int md_fwsectors;   /* firmware sectors */
+ char*md_label;  /* label of the device */
  int md_pad[MDNPAD]; /* padding for future ideas */
 };


This isn't correct on 64-bit platforms.  MDNPAD needs to be 95 on those
platforms.

[...]

Can you report sizeof(md_ioctl) before and after for 32-bit and 64-bit?
I think it may be:
32-bit before: 440
32-bit after:  440
64-bit before: 448
64-bit after:  448

In other words, it looks like it used to produce different sizes on the
different architectures, and still does.  It also looks like 32-bit
before and after and 64-bit before included some undeclared padding
after md_pad, so that this would fail:
CTASSERT(sizeof(md_ioctl) == offsetof(struct md_ioctl, md_pad) +
   sizeof(((struct md_ioctl *)NULL)->md_pad));

In any case a CTASSERT(sizeof(md_ioctl) == XXX) may increase confidence
in the ABI here.


Indeed, the odd count for the padding was nonsense on 64-bit arches.
It followed a uint64_t which gives 64-bit alignment, then 2 ints which
maintain 64-bit alignment.  Then the odd count gets rounded up to even
on 64-bit arcesh.

The padding could be expressed in intmax_t's, but that causes different
problems -- odd padding becomes impossible and char/short/int/long/long
long padding might be needed to prepare for the intmax_t padding.  char
padding is most flexible, but harder to count.

It's surprising how many i386 utilities now work on amd64.  This requires
things like struct layouts for ioctl data to be the same  User (?) pointers
in structs like md_label in the above might be the largest unportability.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322258 - head/sys/kern

[Bruce Evans]

On Thu, 24 Aug 2017, Alan Somers wrote:


On Wed, Aug 9, 2017 at 1:05 AM, Bruce Evans <b...@optusnet.com.au> wrote:

On Tue, 8 Aug 2017, Alan Somers wrote:
...
The compile-time definition of AIO_LISTIO_MAX seems to be broken.  I think
POSIX species that AIO_LISTIO_MAX shall not be defined if the value of
{AIO_LISTIO_MAX} varies at runtime, but it is still defined.  FreeBSD
has this bug for many other sysconf() variables, e.g., {OPEN_MAX}.
Perhaps AIO_LISTIO_MAX is easier to fix since it is not hard-coded as
often as OPEN_MAX.


What you describe is Linux's behavior, but the POSIX requirement is a
bit more general.  All POSIX says is that "The value returned [by
sysconf(3)] shall not be more restrictive than the corresponding value
described to the application when it was compiled with the
implementation's  or ".


No, I described the POSIX requirement.  See the section on limits.h.  It
says that
- {AIO_LISTIO_MAX} is a runtime invariant (so sysctls that change it are
  not POSIX conformant)
- for all runtime invariant limits, the definition shall be omitted from
   if it is unspecified (sic).  This indetermination (sic)
  might depend on the memory size.  [That was not a direct quote
  except for the sic words.  POSIX says "indeterminate" in both places
  in the 1990 and 1996 versions, but this is broken in the 2001 and 2006
  versions.]
So defining AIO_LISTIO_MAX in  is not conformant if you change
it before runtime using something like a tunable.

You quoted the section for sysconf(3).  The wording there is too generic.
It covers both runtime increasable and runtime invariant limits.  It
only really applies to the runtime increasable limits.  For the runtime
invariant, limits it only applies vacuously.  The section on 
disallows defining runtime invariant limits unless they are known at
compile time.  If they would be different at runtime, then they were not
known at comple time, so must not be defined then, and the requirement
that the runtime limits are larger is vacuously satisfied.

There aren't many runtime increasable limits, and at least in the 2006
version, only {OPEN_MAX} is allowed to vary within a process's lifetime.
{OPEN_MAX} can be decreased in practice and the 2006 version doesn't
disallow this provided OPEN_MAX is not defined in  (then
{OPEN_MAX} must be a compile-time invariant).  When it is not defined,
the requirement to only increase it is vacuously satisfied, so only
the special requirement for {OPEN_MAX} applies.  This allows changing
it using setrlimit().  The direction of the change is not limited to
an increase, but many more paragraphs are needed to speciy what happens
when it does decrease (open files above the limit stay open, but
obviously you can't use the current limit to limit the search for these
files...).


...



I dug deeper and found that there wasn't any good reason for the
aio_listio_max limit to exist in the first place.  This DR eliminates
it, which I think will satisfy most of your concerns.
https://reviews.freebsd.org/D12120


This is inconvenient for me to review.

Anything that removes the compile time limit is good.  Except actually
removing it from  would break any applications that use it.
Applications should use the minimum for the limit if they don't care,
else sysconf().

The 2006 version of POSIX says in its section about profiles that most
if its limits are inadequate (so profiles should change them).  I
don't know the mechanism for this.  Just omitting most compile-time
limits and increasing the sysconf() limits won't help for sloppy
applications.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322893 - head/bin/dd

[Bruce Evans]

On Fri, 25 Aug 2017, Conrad Meyer wrote:


On Fri, Aug 25, 2017 at 3:49 PM, Bruce Evans <b...@optusnet.com.au> wrote:

get_off_t() but not the higher level is fixed in my version.


Would you mind incorporating your version of dd into FreeBSD, or
publishing it so someone else can?


I guess it is in my queue, but unfortunately near the end.  My patches
for dd are about 8K (mostly style fixes).

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322893 - head/bin/dd

[Bruce Evans]

On Fri, 25 Aug 2017, Matt Joras wrote:


On 08/25/2017 10:17, Conrad Meyer wrote:

This change seems to break buildworld on MIPS:

/home/cem/head.svn/bin/dd/args.c: In function 'f_bs':
/home/cem/head.svn/bin/dd/args.c:188: warning: format '%zd' expects
type 'signed size_t', but argument 3 has type 'long int'
/home/cem/head.svn/bin/dd/args.c: In function 'f_cbs':
/home/cem/head.svn/bin/dd/args.c:199: warning: format '%zd' expects
type 'signed size_t', but argument 3 has type 'long int'
/home/cem/head.svn/bin/dd/args.c: In function 'f_ibs':
/home/cem/head.svn/bin/dd/args.c:245: warning: format '%zd' expects
type 'signed size_t', but argument 3 has type 'long int'
/home/cem/head.svn/bin/dd/args.c: In function 'f_obs':
/home/cem/head.svn/bin/dd/args.c:266: warning: format '%zd' expects
type 'signed size_t', but argument 3 has type 'long int'

(Yes, it's odd that the SSIZE_MAX constant has 'long' type.)


SSIZE_MAX should have type long, since ssize_t is a long on mips (and
other arches besides i386 and arm).


Actually the reverse.  ssize_t has the correct type int32_t (which
happens to be int) on all 32-bit arches.  SSIZE_MAX shouldn't have
type long on 32-bit arches, but is broken by having that type on arm


Re: the build failure, that's in the GCC C format string checking, so
perhaps it's more accurate to say this breaks the (in-tree) GCC build.
%zd is the right format specifier for ssize_t. I guess GCC's format
string checking is getting confused because SSIZE_MAX is a constant that
expands to type long. Perhaps casting to ssize_t would GCC happier, but
that looks rather wrong.


This is because gcc's format checking actually works.  It detects that
SSIZE_MAX has the incorrect type long on mips because it is defined as
LONG_MAX there.  arm/arm64, powerpc and x86 have ifdefs to define it
correctly as INT_MAX in the 32-bit case.

I finally found where POSIX requires SSIZE_MAX to have the "correct" type
(POSIX doesn't define what that is. but C99 does).  SSIZE_MAX is just in
convered by the same rule as most C99 limits for integer types.

So SSIZE_MAX is not permitted to be what it is on mips.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322893 - head/bin/dd

[Bruce Evans]

On Fri, 25 Aug 2017, Conrad Meyer wrote:


Well, not negative, just large uint64_t numbers that would be negative
as off_t (int64_t).

E.g., dd if=/dev/kmem bs=1 iseek=0xf...foo count=8.  I think we
would like that to work.  I don't recall whether it does or not before
this change.


This is broken on 64-bit systems, first by the horrible get_off_t(),
then by broken range checking in at least old versions of dd.

The first bug in get_off_t() is that it uses strtoimax().  This fails
for offsets >= INTMAX_MAX.  This breaks the natural use of dd to seek
to the kernel part of kmem using dd.  E.g.:

dd if=/dev/kmem bs=1 count=1 iseek=0x802d2100

This should give a negative offset than then works.  (lseek() to negative
offsets is implementation-defined for special files, and FreeBSD defines
it so that it works for seeking in kmem.  This depends on some 2's complement
magic to represent large unsigned offsets as negative signed.)

However, it doesn't work to double bs and halve iseek.  The multiplication
is then done by a higher level.  It exceeds OFF_MAX, so the lseek() isn't
tried.

get_off_t() but not the higher level is fixed in my version.

It does work to calculate the negative offset in another way inside
get_off_t().  E.g., bs=1 iseek=-1 gives -1 which is passed to lseek().
bs=2 iseek=-1 also works to give the correct offset of -2 for lseek().
The arithmetic for converting large number like 0x802d2100 to
a negative value is painful, especially since shells have similar bugs
near INTMAX_MAX and UINTMAX_MAX and with signed and/or unsigned types
even if they support 64-bit integers.

The buggy range checking for bs=2 iseek=0x7fff802d2100 seems to be
only in old versions of dd.  It was in jcl(), and was just for the
initial args (multiplying them would exceed OFF_MAX).  Now there seems
to be no check for overflow.  The multiplication in pos_in() just
overflows even if the check in jcl() passed, later when the seek point
advances past INTMAX_MAX (if this exceeds OFF_MAX, then passing the
result of the multiplicating to lseek() starts overflowing before the
multiplication does).

[Context lost to top posting].

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322893 - head/bin/dd

[Bruce Evans]

On Fri, 25 Aug 2017, Alan Somers wrote:


Log:
 dd(1): Incorrect casting of arguments


It is indeed now incorrect.


 dd(1) casts many of its numeric arguments from uintmax_t to intmax_t and
 back again to detect whether or not the original arguments were negative.
 This is not correct, and causes problems with boundary cases, for example
 when count is SSIZE_MAX-1.


The casts were correct.  Did someone break the range checks?  I wrote most
of the cast, but would rarely write range checks using unportable casts.

SSIZE_MAX-1 isn't a boundary case.  The boundary is at SSIZE_MAX.  %zd
might work for it not.  But %zd is wrong for SSIZE_MAX-1, since the
expression might expand the type.


Modified: head/bin/dd/args.c
==
--- head/bin/dd/args.c  Fri Aug 25 14:42:11 2017(r322892)
+++ head/bin/dd/args.c  Fri Aug 25 15:31:55 2017(r322893)
@@ -41,6 +41,7 @@ __FBSDID("$FreeBSD$");

#include 

+#include 


Unrelated change.


#include 
#include 
#include 
@@ -184,7 +185,7 @@ f_bs(char *arg)

res = get_num(arg);
if (res < 1 || res > SSIZE_MAX)
-   errx(1, "bs must be between 1 and %jd", (intmax_t)SSIZE_MAX);
+   errx(1, "bs must be between 1 and %zd", SSIZE_MAX);
in.dbsz = out.dbsz = (size_t)res;
}



This is incorrect.  The old version used the portable method of casting
a signed value of unknown type to intmax_t before %zd existed.  %zd still
isn't useable in either theory or practice for printing ssize_t's.

Theory:
C99 defines %zd as being for the signed type "corresponding" to size_t.
Who knows what "corresponding" is except for pure 2's complement with
no padding bits?  POSIX only defines ssize_t circularly as being "any
signed integer type capable of representing the range -1 through
SSIZE_MAX" where SSIZE_MAX is the maximum of ssize_t".  It has a few
restrictions that make this not completely circular, but I couldn't
find anywhere where it states the usual restriction on limits -- that
they have the type of the default promotion of the type that they limit.

Practice:
As reported in other replies, some arches define SSIZE_MAX in a way that
makes its type not the default promotion of ssize_t.  Most likely
SSIZE_MAX is bogusly long and ssize_t is int, or vice versa, on an
arch where int and long have the same representation.  Printf format
checking reports this type mismatch because it would be fatal on other
arches.


@@ -195,22 +196,22 @@ f_cbs(char *arg)

res = get_num(arg);
if (res < 1 || res > SSIZE_MAX)
-   errx(1, "cbs must be between 1 and %jd", (intmax_t)SSIZE_MAX);
+   errx(1, "cbs must be between 1 and %zd", SSIZE_MAX);
cbsz = (size_t)res;
}


Similarly.



static void
f_count(char *arg)
{
-   intmax_t res;
+   uintmax_t res;


This breaks the not so careful checking for negative args.

dd wants to allow negative args for seek offsets and not much else.
It has a badly written get_off_t() to handle the offsets, but old code
like this still used the old function get_num() that returns an unsigned
type.  This is a feature since get_off_t() is so badly written that it
is better to convert get_num() almost as done here.  My version does this
internally:

X /*
X  * Convert an expression of the above forms to an off_t.  This version does
X  * not handle negative numbers perfectly.  It assumes 2's complement and
X  * maybe nonnegative multipliers.  I hope perfect handling is not necessary
X  * for dd.
X  */
X static off_t
X get_off_t(const char *val)
X {
X   u_quad_t num;
X 
X 	num = get_num(val);

X   if (num != (u_quad_t)(off_t)num)
X   errx(1, "%s: offset too large", oper);
X   return ((off_t)num);
X }

This still uses the bogus type u_quad_t since that is what get_num() returns
in the old version of dd that this patch is for.

There should be no restriction except UINTMAX_MAX on counts.  This is fixed
in my version:

X static void
X f_count(char *arg)
X {
X 
X 	cpy_cnt = get_num(arg);

X #if 0
X   if (cpy_cnt == 0)
X   terminate(0);
X #endif
X }

get_num() already did correct range checking.

A count of 0 should mean infinity.

Code elsewhere needs be careful to go to infinity for count 0 and to
not multiply a large count by a block size.  In practice, counts of
nearly UINTMAX_MAX are unreachable.



-   res = (intmax_t)get_num(arg);
-   if (res < 0)
-   errx(1, "count cannot be negative");
+   res = get_num(arg);
+   if (res == UINTMAX_MAX)
+   errc(1, ERANGE, "%s", oper);


This is nonsense error checking.  get_num() already did correct checking
and exited on range errors.  A value of UINTMAX_MAX is not necessarily an
error.  After calling strtoumax(), it might be either a range error or at
the end of the range (including the negative end).  Since get_num() returned,
it was not a range error.  The check here does extra work to 

svn commit: r322884 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Fri Aug 25 10:57:17 2017
New Revision: 322884
URL: https://svnweb.freebsd.org/changeset/base/322884

Log:
  Fix bugs in (mostly) not-yet-activated parts of early/emergency output:
  
  - map the hard-coded frame buffer address above KERNBASE.  Using the
physical address only worked because of larger mapping bugs.
  
The hard-coded frame buffer address only works on x86.  Use messy ifdefs
to try to avoid warnings about unused code for other arches.
  
  - remove the sysctl for reading and writing the table kernel console
attributes.  Writing only worked for emergency output since normal
output uses unalterd copies.
  
  - fix the test for the emergency console being usable
  
  - explain why a hard-coded attribute is used very early.  Emergency output
works on x86 even before the pcpu pointer is initialized.

Modified:
  head/sys/dev/syscons/syscons.c

Modified: head/sys/dev/syscons/syscons.c
==
--- head/sys/dev/syscons/syscons.c  Fri Aug 25 09:51:22 2017
(r322883)
+++ head/sys/dev/syscons/syscons.c  Fri Aug 25 10:57:17 2017
(r322884)
@@ -76,6 +76,13 @@ __FBSDID("$FreeBSD$");
 #endif
 #include 
 
+#if defined(__amd64__) || defined(__i386__)
+#include 
+
+#include 
+#include 
+#endif
+
 #include 
 #include 
 #include 
@@ -137,8 +144,6 @@ static  int sc_no_suspend_vtswitch = 0;
 static int sc_susp_scr;
 
 static SYSCTL_NODE(_hw, OID_AUTO, syscons, CTLFLAG_RD, 0, "syscons");
-SYSCTL_OPAQUE(_hw_syscons, OID_AUTO, kattr, CTLFLAG_RW,
-_kattrtab, sizeof(sc_kattrtab), "CU", "kernel console attributes");
 static SYSCTL_NODE(_hw_syscons, OID_AUTO, saver, CTLFLAG_RD, 0, "saver");
 SYSCTL_INT(_hw_syscons_saver, OID_AUTO, keybonly, CTLFLAG_RW,
 _saver_keyb_only, 0, "screen saver interrupted by input only");
@@ -277,16 +282,17 @@ ec_putc(int c)
if (sc_console == NULL) {
 #if !defined(__amd64__) && !defined(__i386__)
return;
-#endif
+#else
/*
 * This is enough for ec_putc() to work very early on x86
 * if the kernel starts in normal color text mode.
 */
-   fb = 0xb8000;
+   fb = KERNBASE + 0xb8000;
xsize = 80;
ysize = 25;
+#endif
} else {
-   if (main_console.status & GRAPHICS_MODE)
+   if (!ISTEXTSC(_console))
return;
fb = main_console.sc->adp->va_window;
xsize = main_console.xsize;
@@ -4159,7 +4165,7 @@ static int
 sc_kattr(void)
 {
 if (sc_console == NULL)
-   return (SC_KERNEL_CONS_ATTR);
+   return (SC_KERNEL_CONS_ATTR);   /* for very early, before pcpu */
 return (sc_kattrtab[PCPU_GET(cpuid) % nitems(sc_kattrtab)]);
 }
 
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322878 - in head: sys/dev/syscons sys/sys usr.sbin/vidcontrol

[Bruce Evans]

Author: bde
Date: Fri Aug 25 07:04:41 2017
New Revision: 322878
URL: https://svnweb.freebsd.org/changeset/base/322878

Log:
  Support setting the colors of cursors for the VGA renderer.
  
  Advertise this by changing the defaults to mostly red.  If you don't like
  this, change them (almost) back using:
 vidcontrol -c charcolors,base=7,height=0
 vidcontrol -c mousecolors,base=0[,height=15]
  
  The (graphics mode only) mouse cursor colors were hard-coded to a black
  border and lightwhite interior.  Black for the border is the worst
  possible default, since it is the same as the default black background
  and not good for any dark background.  Reversing this gives the better
  default of X Windows.  Coloring everything works better still.  Now
  the coloring defaults to a lightwhite border and red interior.
  
  Coloring for the character cursor is more complicated and mode
  dependent.  The new coloring doesn't apply for hardware cursors.  For
  non-block cursors, it only applies in graphics mode.  In text mode,
  the cursor color was usually a hard-coded (dull)white for the background
  only, unless the foreground was white when it was a hard-coded black
  for the background only, unless the foreground was white and the
  background was black it was reverse video.  In graphics mode, it was
  always reverse video for the block cursor.  Reverse video is worse,
  especially over cutmarking regions, since cutmarking still uses simple
  reverse video (nothing better is possible in text mode) and double
  reverse video for the cursor gives normal video.  Now, graphics mode
  uses the same algorithm as the best case for text mode in all cases
  for graphics mode.  The hard-coded sequence { white, black, } for the
  background is now { red, white, blue, } where the first 2 colors can
  be configured.  The blue color at the end is a sentinel which prevents
  reverse video being used in most cases but breaks the compatibility
  setting for white on black and black on white characters.  This will
  be fixed later.  The compatibility setting is most needed for mono modes.
  
  The previous commit to syscons.c changed sc_cnterm() to be more careful.
  It followed null pointers in some cases.  But sc_cnterm() has been
  unreachable for 15+ years since changes for multiple consoles turned
  off calls to the the cnterm destructor for all console drivers.  Before
  them, it was only called at boot time.  So no driver with an attached
  console has ever been unloadable and not even the non-console destructors
  have been tested much.

Modified:
  head/sys/dev/syscons/scvgarndr.c
  head/sys/dev/syscons/syscons.c
  head/sys/dev/syscons/syscons.h
  head/sys/sys/consio.h
  head/usr.sbin/vidcontrol/vidcontrol.1
  head/usr.sbin/vidcontrol/vidcontrol.c

Modified: head/sys/dev/syscons/scvgarndr.c
==
--- head/sys/dev/syscons/scvgarndr.cFri Aug 25 05:49:37 2017
(r322877)
+++ head/sys/dev/syscons/scvgarndr.cFri Aug 25 07:04:41 2017
(r322878)
@@ -356,32 +356,28 @@ vga_flipattr(u_short a, int blink)
 }
 
 static u_short
-vga_cursorattr_adj(u_short a, int blink)
+vga_cursorattr_adj(scr_stat *scp, u_short a, int blink)
 {
-   /*
-* !blink means pixel mode, and the cursor attribute in that case
-* is simplistic reverse video.
-*/
-   if (!blink)
-   return (vga_flipattr(a, blink));
+   int i;
+   u_short bg, bgmask, fg, newbg;
 
/*
 * The cursor attribute is usually that of the underlying char
-* with the bg changed to white.  If the bg is already white,
-* then the bg is changed to black.  The fg is usually not
-* changed, but if it is the same as the new bg then it is
-* changed to the inverse of the new bg.
+* with only the bg changed, to the first preferred color that
+* differs from both the fg and bg.  If there is no such color,
+* use reverse video.
 */
-   if ((a & 0x7000) == 0x7000) {
-   a &= 0x8f00;
-   if ((a & 0x0700) == 0)
-   a |= 0x0700;
-   } else {
-   a |= 0x7000;
-   if ((a & 0x0700) == 0x0700)
-   a &= 0xf000;
+   bgmask = blink ? 0x7000 : 0xf000;
+   bg = a & bgmask;
+   fg = a & 0x0f00;
+   for (i = 0; i < nitems(scp->curs_attr.bg); i++) {
+   newbg = (scp->curs_attr.bg[i] << 12) & bgmask;
+   if (newbg != bg && newbg != (fg << 4))
+   break;
}
-   return (a);
+   if (i == nitems(scp->curs_attr.bg))
+   return (vga_flipattr(a, blink));
+   return (fg | newbg | (blink ? a & 0x8000 : 0));
 }
 
 static void
@@ -522,6 +518,12 @@ draw_txtcharcursor(scr_stat *scp, int at, u_short c, u
return;
if (flip)
a = vga_flipattr(a, TRUE);
+  

svn commit: r322870 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Fri Aug 25 02:41:01 2017
New Revision: 322870
URL: https://svnweb.freebsd.org/changeset/base/322870

Log:
  Oops, the previous commit was missing 1 line.

Modified:
  head/sys/dev/syscons/syscons.c

Modified: head/sys/dev/syscons/syscons.c
==
--- head/sys/dev/syscons/syscons.c  Fri Aug 25 02:37:32 2017
(r322869)
+++ head/sys/dev/syscons/syscons.c  Fri Aug 25 02:41:01 2017
(r322870)
@@ -1933,6 +1933,7 @@ sc_cnputc(struct consdev *cd, int c)
 u_char buf[1];
 scr_stat *scp = sc_console;
 void *oldts, *ts;
+struct sc_term_sw *oldtsw;
 #ifndef SC_NO_HISTORY
 #if 0
 struct tty *tp;
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322869 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Fri Aug 25 02:37:32 2017
New Revision: 322869
URL: https://svnweb.freebsd.org/changeset/base/322869

Log:
  Fix missing switching of the terminal emulator when switching the
  terminal state for kernel console output.
  
  r56043 in 2000 added many complications to support dynamic selection
  of the terminal emulator using modules and the ioctl CONS_SETTERM.
  This was never completed.  There are still no modules, but it is easy
  to restore the scterm and dumb emulators at compile time.  Then
  boot-time configuration for the preferred one doesn't work right, but
  CONS_SETTERM almost works after fixing this bug.  CONS_SETTERM only
  switches the emulator for the user state, leaving the kernel state(s)
  still using the boot-time emulator.  The fix is especially important
  when switching from sc to scteken, since the scteken state has pointers
  in it.
  
  Rename kernel_console_ts to sc_kts.

Modified:
  head/sys/dev/syscons/syscons.c

Modified: head/sys/dev/syscons/syscons.c
==
--- head/sys/dev/syscons/syscons.c  Fri Aug 25 00:28:56 2017
(r322868)
+++ head/sys/dev/syscons/syscons.c  Fri Aug 25 02:37:32 2017
(r322869)
@@ -98,7 +98,8 @@ staticint sc_console_unit = -1;
 static int sc_saver_keyb_only = 1;
 static  scr_stat   *sc_console;
 static  struct consdev *sc_consptr;
-static void*kernel_console_ts[MAXCPU];
+static void*sc_kts[MAXCPU];
+static struct sc_term_sw *sc_ktsw;
 static scr_statmain_console;
 static struct tty  *main_devs[MAXCONS];
 
@@ -564,6 +565,7 @@ sc_attach_unit(int unit, int flags)
scinit(unit, flags);
 
if (sc_console->tsw->te_size > 0) {
+   sc_ktsw = sc_console->tsw;
/* assert(sc_console->ts != NULL); */
oldts = sc_console->ts;
for (i = 0; i <= mp_maxid; i++) {
@@ -573,7 +575,7 @@ sc_attach_unit(int unit, int flags)
sc_console->ts = ts;
(*sc_console->tsw->te_default_attr)(sc_console, sc_kattrtab[i],
SC_KERNEL_CONS_REV_ATTR);
-   kernel_console_ts[i] = ts;
+   sc_kts[i] = ts;
}
sc_console->ts = oldts;
(*sc_console->tsw->te_default_attr)(sc_console, SC_NORM_ATTR,
@@ -1728,11 +1730,14 @@ sc_cnterm(struct consdev *cp)
 sccnupdate(sc_console);
 #endif
 
-for (i = 0; i <= mp_maxid; i++) {
-   ts = kernel_console_ts[i];
-   kernel_console_ts[i] = NULL;
-   (*sc_console->tsw->te_term)(sc_console, );
-   free(ts, M_DEVBUF);
+if (sc_ktsw != NULL) {
+   for (i = 0; i <= mp_maxid; i++) {
+   ts = sc_kts[i];
+   sc_kts[i] = NULL;
+   (*sc_ktsw->te_term)(sc_console, );
+   free(ts, M_DEVBUF);
+   }
+   sc_ktsw = NULL;
 }
 scterm(sc_console_unit, SC_KERNEL_CONSOLE);
 sc_console_unit = -1;
@@ -1992,13 +1997,16 @@ sc_cnputc(struct consdev *cd, int c)
sizeof(sc_cnputc_log))
continue;
/* Console output has a per-CPU "input" state.  Switch for it. */
+   oldtsw = scp->tsw;
oldts = scp->ts;
-   ts = kernel_console_ts[PCPU_GET(cpuid)];
+   ts = sc_kts[PCPU_GET(cpuid)];
if (ts != NULL) {
+   scp->tsw = sc_ktsw;
scp->ts = ts;
(*scp->tsw->te_sync)(scp);
}
sc_puts(scp, buf, 1);
+   scp->tsw = oldtsw;
scp->ts = oldts;
(*scp->tsw->te_sync)(scp);
 }
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322709 - in head/sys: dev/syscons sys

[Bruce Evans]

Author: bde
Date: Sat Aug 19 23:13:33 2017
New Revision: 322709
URL: https://svnweb.freebsd.org/changeset/base/322709

Log:
  Fix setting of defaults for the text cursor.
  
  There was already a per-vty defaults field, but it was useless since it was
  only initialized when propagating the global settings and thus no different
  from the current global settings and not per-vty.  The global defaults field
  was also invariant after boot time, but not quite so useless.
  
  Fix this by adding a second selection bit the the control flags of the
  relevant ioctl().  vidcontrol doesn't support this yet.  Setting either
  default propagates the change to the current setting for the same level
  and then to all lower levels.
  
  Improve the 3-way escape sequence used by termcap to control the cursor.
  The "normal" (ve) case has always used reset, so the user could set
  it to anything, but since the reset is to a global value this is not
  very useful, especially since the "very visible" (vs) case doesn't
  reset but inconsistently forces to a blinking block.  Change vs to
  first reset and then XOR the blinking bit so that it is predictably
  different from ve.

Modified:
  head/sys/dev/syscons/scterm-teken.c
  head/sys/dev/syscons/syscons.c
  head/sys/sys/consio.h

Modified: head/sys/dev/syscons/scterm-teken.c
==
--- head/sys/dev/syscons/scterm-teken.c Sat Aug 19 21:40:42 2017
(r322708)
+++ head/sys/dev/syscons/scterm-teken.c Sat Aug 19 23:13:33 2017
(r322709)
@@ -684,7 +684,6 @@ scteken_param(void *arg, int cmd, unsigned int value)
static int tcattrs[] = {
CONS_RESET_CURSOR | CONS_LOCAL_CURSOR,  /* normal */
CONS_HIDDEN_CURSOR | CONS_LOCAL_CURSOR, /* invisible */
-   CONS_BLINK_CURSOR | CONS_LOCAL_CURSOR,  /* very visible */
};
scr_stat *scp = arg;
int flags, n, v0, v1, v2;
@@ -727,6 +726,13 @@ scteken_param(void *arg, int cmd, unsigned int value)
case TP_SETLOCALCURSOR:
if (value < sizeof(tcattrs) / sizeof(tcattrs[0]))
sc_change_cursor_shape(scp, tcattrs[value], -1, -1);
+   else if (value == 2) {
+   sc_change_cursor_shape(scp,
+   CONS_RESET_CURSOR | CONS_LOCAL_CURSOR, -1, -1);
+   flags = scp->base_curs_attr.flags ^ CONS_BLINK_CURSOR;
+   sc_change_cursor_shape(scp,
+   flags | CONS_LOCAL_CURSOR, -1, -1);
+   }
break;
case TP_SHOWCURSOR:
if (value != 0)

Modified: head/sys/dev/syscons/syscons.c
==
--- head/sys/dev/syscons/syscons.c  Sat Aug 19 21:40:42 2017
(r322708)
+++ head/sys/dev/syscons/syscons.c  Sat Aug 19 23:13:33 2017
(r322709)
@@ -943,13 +943,19 @@ sctty_ioctl(struct tty *tp, u_long cmd, caddr_t data, 
return 0;
 
 case CONS_GETCURSORSHAPE:   /* get cursor shape (new interface) */
-   switch (((int *)data)[0] & CONS_LOCAL_CURSOR) {
+   switch (((int *)data)[0] & (CONS_DEFAULT_CURSOR | CONS_LOCAL_CURSOR)) {
case 0:
cap = >curs_attr;
break;
case CONS_LOCAL_CURSOR:
cap = >base_curs_attr;
break;
+   case CONS_DEFAULT_CURSOR:
+   cap = >dflt_curs_attr;
+   break;
+   case CONS_DEFAULT_CURSOR | CONS_LOCAL_CURSOR:
+   cap = >dflt_curs_attr;
+   break;
}
((int *)data)[1] = cap->base;
((int *)data)[2] = cap->height;
@@ -3030,7 +3036,10 @@ change_cursor_shape(scr_stat *scp, int flags, int base
 
 if (flags & CONS_RESET_CURSOR)
scp->base_curs_attr = scp->dflt_curs_attr;
-else
+else if (flags & CONS_DEFAULT_CURSOR) {
+   sc_adjust_ca(>dflt_curs_attr, flags, base, height);
+   scp->base_curs_attr = scp->dflt_curs_attr;
+} else
sc_adjust_ca(>base_curs_attr, flags, base, height);
 
 if ((scp == scp->sc->cur_scp) && !ISGRAPHSC(scp)) {
@@ -3062,7 +3071,10 @@ sc_change_cursor_shape(scr_stat *scp, int flags, int b
 sc = scp->sc;
 if (flags & CONS_RESET_CURSOR)
sc->curs_attr = sc->dflt_curs_attr;
-else
+else if (flags & CONS_DEFAULT_CURSOR) {
+   sc_adjust_ca(>dflt_curs_attr, flags, base, height);
+   sc->curs_attr = sc->dflt_curs_attr;
+} else
sc_adjust_ca(>curs_attr, flags, base, height);
 
 for (i = sc->first_vty; i < sc->first_vty + sc->vtys; ++i) {

Modified: head/sys/sys/consio.h
==
--- head/sys/sys/consio.h   Sat Aug 19 21:40:42 2017(r322708)
+++ head/sys/sys/consio.h   Sat Aug 19 23:13:33 2017(r322709)
@@ -187,6 +187,7 @@ typedef struct mouse_info mouse_info_t;
 #define CONS_HIDDEN_CURSOR 

svn commit: r322708 - in head/sys: dev/syscons sys

[Bruce Evans]

Author: bde
Date: Sat Aug 19 21:40:42 2017
New Revision: 322708
URL: https://svnweb.freebsd.org/changeset/base/322708

Log:
  Rename curr_curs_attr to base_curr_attr.  The actual current cursor
  attribute field is curs_attr.  The base field holds user data translated
  in a reversible way and is needed because current field holds this in
  an irreversible way for efficiency.
  
  Factor out some common code for the reversible translation.  This is
  slightly simpler now, and much easier to expand.
  
  Translate the magic flags value -1 to a single control flag internally
  up front so other flags can be trusted later.  This can be used for the
  relevant ioctl() too.
  
  Remove CONS_CURSOR_FLAGS which contained all the control flags.  It was
  unused and not useful.  After adding more flags, there will be tests on
  a couple at a time but never on them all.  This API should have used this
  to disallow unknown flags.

Modified:
  head/sys/dev/syscons/scterm-teken.c
  head/sys/dev/syscons/syscons.c
  head/sys/dev/syscons/syscons.h
  head/sys/sys/consio.h

Modified: head/sys/dev/syscons/scterm-teken.c
==
--- head/sys/dev/syscons/scterm-teken.c Sat Aug 19 21:17:53 2017
(r322707)
+++ head/sys/dev/syscons/scterm-teken.c Sat Aug 19 21:40:42 2017
(r322708)
@@ -730,9 +730,9 @@ scteken_param(void *arg, int cmd, unsigned int value)
break;
case TP_SHOWCURSOR:
if (value != 0)
-   flags = scp->curr_curs_attr.flags & ~CONS_HIDDEN_CURSOR;
+   flags = scp->base_curs_attr.flags & ~CONS_HIDDEN_CURSOR;
else
-   flags = scp->curr_curs_attr.flags | CONS_HIDDEN_CURSOR;
+   flags = scp->base_curs_attr.flags | CONS_HIDDEN_CURSOR;
sc_change_cursor_shape(scp, flags | CONS_LOCAL_CURSOR, -1, -1);
break;
case TP_SWITCHVT:

Modified: head/sys/dev/syscons/syscons.c
==
--- head/sys/dev/syscons/syscons.c  Sat Aug 19 21:17:53 2017
(r322707)
+++ head/sys/dev/syscons/syscons.c  Sat Aug 19 21:40:42 2017
(r322708)
@@ -873,6 +873,7 @@ sctty_ioctl(struct tty *tp, u_long cmd, caddr_t data, 
 {
 int error;
 int i;
+struct cursor_attr *cap;
 sc_softc_t *sc;
 scr_stat *scp;
 int s;
@@ -942,15 +943,17 @@ sctty_ioctl(struct tty *tp, u_long cmd, caddr_t data, 
return 0;
 
 case CONS_GETCURSORSHAPE:   /* get cursor shape (new interface) */
-   if (((int *)data)[0] & CONS_LOCAL_CURSOR) {
-   ((int *)data)[0] = scp->curr_curs_attr.flags;
-   ((int *)data)[1] = scp->curr_curs_attr.base;
-   ((int *)data)[2] = scp->curr_curs_attr.height;
-   } else {
-   ((int *)data)[0] = sc->curs_attr.flags;
-   ((int *)data)[1] = sc->curs_attr.base;
-   ((int *)data)[2] = sc->curs_attr.height;
+   switch (((int *)data)[0] & CONS_LOCAL_CURSOR) {
+   case 0:
+   cap = >curs_attr;
+   break;
+   case CONS_LOCAL_CURSOR:
+   cap = >base_curs_attr;
+   break;
}
+   ((int *)data)[1] = cap->base;
+   ((int *)data)[2] = cap->height;
+   ((int *)data)[0] = cap->flags;
return 0;
 
 case CONS_SETCURSORSHAPE:   /* set cursor shape (new interface) */
@@ -2982,15 +2985,15 @@ update_cursor_image(scr_stat *scp)
 void
 sc_set_cursor_image(scr_stat *scp)
 {
-scp->curs_attr.flags = scp->curr_curs_attr.flags;
+scp->curs_attr = scp->base_curs_attr;
 if (scp->curs_attr.flags & CONS_HIDDEN_CURSOR) {
/* hidden cursor is internally represented as zero-height underline */
scp->curs_attr.flags = CONS_CHAR_CURSOR;
scp->curs_attr.base = scp->curs_attr.height = 0;
 } else if (scp->curs_attr.flags & CONS_CHAR_CURSOR) {
-   scp->curs_attr.base = imin(scp->curr_curs_attr.base,
+   scp->curs_attr.base = imin(scp->base_curs_attr.base,
  scp->font_size - 1);
-   scp->curs_attr.height = imin(scp->curr_curs_attr.height,
+   scp->curs_attr.height = imin(scp->base_curs_attr.height,
scp->font_size - scp->curs_attr.base);
 } else {   /* block cursor */
scp->curs_attr.base = 0;
@@ -3005,19 +3008,30 @@ sc_set_cursor_image(scr_stat *scp)
 }
 
 static void
+sc_adjust_ca(struct cursor_attr *cap, int flags, int base, int height)
+{
+if (0) {
+   /* Dummy clause to avoid changing indentation later. */
+} else {
+   if (base >= 0)
+   cap->base = base;
+   if (height >= 0)
+   cap->height = height;
+   if (!(flags & CONS_SHAPEONLY_CURSOR))
+   cap->flags = flags & CONS_CURSOR_ATTRS;
+}
+}
+
+static void
 change_cursor_shape(scr_stat *scp, int flags, int base, int height)
 {
 if ((scp == 

svn commit: r322705 - in head/sys: arm/arm dev/syscons isa mips/mips powerpc/powerpc

[Bruce Evans]

Author: bde
Date: Sat Aug 19 19:33:16 2017
New Revision: 322705
URL: https://svnweb.freebsd.org/changeset/base/322705

Log:
  Use better hard-coded defaults for the cursor shape, and remove nearby
  redundant initializations.
  
  Hard-code base = 0, height = (approx. 1/8 of the boot-time font height)
  in all cases, and remove the BIOS/MD support for setting these values.
  This asks for an underline cursor sized for the boot-time font instead
  of various less hard-coded but worse values.  I used that think that
  the x86 BIOS always gave the same values as the above hard-coding, but
  on 1 of my systems it gives the wrong value of base = 1.
  
  The remaining BIOS fields are shift_state and bell_pitch.  These are now
  consistently not explicitly reinitialized to 0.  All sc_get_bios_value()
  functions except x86's are now empty, and the only useful thing that x86
  returns is shift_state.  This really belongs in atkbdc, but heavier
  use of the BIOS to read the more useful typematic rate has been removed
  there.  fb still makes much heavier use of the BIOS.

Modified:
  head/sys/arm/arm/sc_machdep.c
  head/sys/dev/syscons/syscons.c
  head/sys/dev/syscons/syscons.h
  head/sys/isa/syscons_isa.c
  head/sys/mips/mips/sc_machdep.c
  head/sys/powerpc/powerpc/sc_machdep.c

Modified: head/sys/arm/arm/sc_machdep.c
==
--- head/sys/arm/arm/sc_machdep.c   Sat Aug 19 19:06:40 2017
(r322704)
+++ head/sys/arm/arm/sc_machdep.c   Sat Aug 19 19:33:16 2017
(r322705)
@@ -78,9 +78,6 @@ sc_get_softc(int unit, int flags)
 void
 sc_get_bios_values(bios_values_t *values)
 {
-   values->cursor_start = 0;
-   values->cursor_end = 32;
-   values->shift_state = 0;
 }
 
 int

Modified: head/sys/dev/syscons/syscons.c
==
--- head/sys/dev/syscons/syscons.c  Sat Aug 19 19:06:40 2017
(r322704)
+++ head/sys/dev/syscons/syscons.c  Sat Aug 19 19:33:16 2017
(r322705)
@@ -3208,14 +3208,8 @@ scinit(int unit, int flags)
scp->cursor_pos = scp->cursor_oldpos = row*scp->xsize + col;
(*scp->tsw->te_sync)(scp);
 
-   /* Sync BIOS cursor shape to s/w (sc only). */
-   if (bios_value.cursor_end < scp->font_size)
-   sc->dflt_curs_attr.base = scp->font_size - 
- bios_value.cursor_end - 1;
-   else
-   sc->dflt_curs_attr.base = 0;
-   i = bios_value.cursor_end - bios_value.cursor_start + 1;
-   sc->dflt_curs_attr.height = imin(i, scp->font_size);
+   sc->dflt_curs_attr.base = 0;
+   sc->dflt_curs_attr.height = howmany(scp->font_size, 8);
sc->dflt_curs_attr.flags = 0;
sc->curs_attr = sc->dflt_curs_attr;
scp->curr_curs_attr = scp->dflt_curs_attr = sc->curs_attr;

Modified: head/sys/dev/syscons/syscons.h
==
--- head/sys/dev/syscons/syscons.h  Sat Aug 19 19:06:40 2017
(r322704)
+++ head/sys/dev/syscons/syscons.h  Sat Aug 19 19:33:16 2017
(r322705)
@@ -516,8 +516,6 @@ typedef struct sc_renderer {
   SI_SUB_DRIVERS, SI_ORDER_MIDDLE)
 
 typedef struct {
-   int cursor_start;
-   int cursor_end;
int shift_state;
int bell_pitch;
 } bios_values_t;

Modified: head/sys/isa/syscons_isa.c
==
--- head/sys/isa/syscons_isa.c  Sat Aug 19 19:06:40 2017(r322704)
+++ head/sys/isa/syscons_isa.c  Sat Aug 19 19:33:16 2017(r322705)
@@ -207,17 +207,11 @@ sc_get_bios_values(bios_values_t *values)
 #if defined(__i386__) || defined(__amd64__)
uint8_t shift;
 
-   values->cursor_start = *(uint8_t *)BIOS_PADDRTOVADDR(0x461);
-   values->cursor_end = *(uint8_t *)BIOS_PADDRTOVADDR(0x460);
shift = *(uint8_t *)BIOS_PADDRTOVADDR(0x417);
values->shift_state = ((shift & BIOS_CLKED) != 0 ? CLKED : 0) |
((shift & BIOS_NLKED) != 0 ? NLKED : 0) |
((shift & BIOS_SLKED) != 0 ? SLKED : 0) |
((shift & BIOS_ALKED) != 0 ? ALKED : 0);
-#else
-   values->cursor_start = 0;
-   values->cursor_end = 32;
-   values->shift_state = 0;
 #endif
values->bell_pitch = BELL_PITCH;
 }

Modified: head/sys/mips/mips/sc_machdep.c
==
--- head/sys/mips/mips/sc_machdep.c Sat Aug 19 19:06:40 2017
(r322704)
+++ head/sys/mips/mips/sc_machdep.c Sat Aug 19 19:33:16 2017
(r322705)
@@ -78,9 +78,6 @@ sc_get_softc(int unit, int flags)
 void
 sc_get_bios_values(bios_values_t *values)
 {
-   values->cursor_start = 0;
-   values->cursor_end = 32;
-   values->shift_state = 0;
 }
 
 int

Modified: 

svn commit: r322693 - head/usr.sbin/vidcontrol

[Bruce Evans]

Author: bde
Date: Sat Aug 19 12:14:46 2017
New Revision: 322693
URL: https://svnweb.freebsd.org/changeset/base/322693

Log:
  Reduce complexity and backwards compatibilty a little by removing new aliases
  and repurposing "blink".  Improve accuracy of documentation of historical
  mistakes and other bugs.
  
  "blink" now means "set the blink attribute for the target(s)" instead of
  "set the blink attribute and clear other attributes [and control flags]".
  It was even more confusing to use "blinking" for the single attribute to
  keep the old meaning for "blink".
  
  "destructive" is not as historically broken or gone as the previous version
  said.
  
  The bugs involving resetting from defaults are now understood and partly
  documented (the defaults are mis-initialized).

Modified:
  head/usr.sbin/vidcontrol/vidcontrol.1
  head/usr.sbin/vidcontrol/vidcontrol.c

Modified: head/usr.sbin/vidcontrol/vidcontrol.1
==
--- head/usr.sbin/vidcontrol/vidcontrol.1   Sat Aug 19 09:38:43 2017
(r322692)
+++ head/usr.sbin/vidcontrol/vidcontrol.1   Sat Aug 19 12:14:46 2017
(r322693)
@@ -137,26 +137,38 @@ The following override
 .Cm setting Ns s
 are available:
 .Bl -tag -width indent
-.It Cm block
+.It Cm normal
 Set to a block covering 1 character cell,
 with a configuration-dependent coloring
 that should be at worst inverse video.
-.It Cm blinkingblock
-Set to a blinking
-.Cm block .
-.It Cm underline
-Set to
+.It Cm destructive
+Set to a blinking sub-block with
 .Cm height
 scanlines starting at
 .Cm base .
-This only gives an underline in normal configurations.
-.It Cm blinkingunderline
-Set to a blinking
-.Cm underline .
+The name
+.Dq destructive
+is bad for backwards compatibility.
+This
+.Cm setting
+should not force destructiveness,
+and it now only gives destructiveness in some
+configurations (typically for hardware cursors
+in text mode).
+Blinking limits destructiveness.
+This
+.Cm setting
+should now be spelled
+.Cm normal , Ns Cm blink , Ns Cm noblock .
+A non-blinking destructive cursor would be unusable,
+so old versions of
+.Nm
+didn't support it,
+and this version doesn't have an override for it.
 .It Cm base Ns = Ns Ar value, Cm height Ns = Ns Ar value
 Set the specified scanline parameters.
-These parameters are only active in so-called
-.Cm char ( underline )
+These parameters are only active in
+.Cm noblock
 mode.
 .Cm value
 is an integer in any base supported by
@@ -164,7 +176,7 @@ is an integer in any base supported by
 Setting
 .Cm height
 to 0 turns off the cursor in
-.Cm char
+.Cm noblock
 mode.
 Negative
 .Ar value Ns s
@@ -174,73 +186,57 @@ Positive
 are clamped to fit in the character cell when the cursor is drawn.
 .El
 .Pp
-The following override
-.Cm setting Ns s
-are provided for backwards compatibility:
-.Bl -tag -width indent
-.It Cm normal
-Old name for
-.Cm block .
-.It Cm blink
-Old name for
-.Cm blinkingblock .
-.It Cm destructive
-Bad old name for
-.Cm blinkingunderline .
-The so-called
-.Cm destructive
-cursor was only destructive due to a bug that was fixed in
-.Fx 4.1.0 .
-The destruction in the non-blinking case was so large that
-this case was not supported by
-.Nm ,
-so no backwards compatible name is needed for this case.
-.El
-.Pp
 The following modifier
 .Cm setting Ns s
 are available:
 .Bl -tag -width indent
-.It Cm blinking , noblinking
+.It Cm blink , noblink
 Set or clear the blinking attribute.
-.It Cm char , nochar
-Set or clear the so-called
-.Cm char
+This is not quite backwards compatible.
+In old versions of
+.Nm , Cm blink
+was an override to a blinking block.
+.It Cm block , noblock
+Set or clear the
+.Cm block
 attribute.
-Names in this section follow the bad old names of
-flags in the implementation.
-.Cm char
-now means
-.Do
-activate the scanline parameters,
-and implement them using the best available method
-(at worst, actually use the old
-.Cm char
-method with its destruction fixed if it is configured,
-else silently ignore this flag).
-.Dc
+This attribute is the inverse of the flag
+.Dv CONS_CHAR_CURSOR
+in the implementation.
+It deactivates the scanline parameters,
+and expresses a preference for using a
+simpler method of implementation.
+Its inverse does the opposite.
+When the scanline parameters give a full block,
+this attribute reduces to a method selection bit.
+The
+.Cm block
+method tends to give better coloring.
 .It Cm hidden , nohidden
 Set or clear the hidden attribute.
+.El
+.Pp
+The following (non-sticky) flags control application of the
+.Cm setting Ns s :
+.Bl -tag -width indent
 .It Cm local
-Apply the changes (except for the
-.Cm local
-and
-.Cm reset
-control flags)
-to the current vty.
-The default is to apply them to all vtys.
+Apply the changes to the current vty.
+The default is to apply them to a global place
+and copy from there to all vtys.
 .It Cm reset
 Reset everything.
 The default is to not reset.
 When the
 

svn commit: r322662 - in head/sys: dev/syscons dev/vt teken

[Bruce Evans]

Author: bde
Date: Fri Aug 18 15:40:40 2017
New Revision: 322662
URL: https://svnweb.freebsd.org/changeset/base/322662

Log:
  Fix syscons escape sequence for setting the local cursor type.  This sequence
  was aliased to a vt sequence, causing and fixing various bugs.
  
  For syscons, this restores support for arg 2 which sets blinking block
  too forcefully, and restores bugs for arg 0 and 1.  Arg 2 is used for
  vs in the cons25 entry in termcap, but I've never noticed an application
  that uses this.  The bugs involve replacing local settings by global
  ones and need better handling of defaults to fix.
  
  For vt, this requires moving the aliasing code from teken to vt where
  it belongs.  This sequences is very important for cons25 compatibility
  in vt since it is used by the cons25 termcap entries for ve, vi and
  vs.  vt can't properly support vs for either cons25 or xterm since it
  doesn't support blinking.  For xterm, the termcap entry for vs asks
  for something different using 12;25h instead of 25h.
  
  Rename C25CURS for this to C25LCT and change its description to be closer
  to echoing the old comment about it.  CURS is too generic.
  
  Fix missing syscons escape sequence for setting the global cursor shape
  (and type).  Only support this in syscons since vt can't emulate anything
  in it.

Modified:
  head/sys/dev/syscons/scterm-teken.c
  head/sys/dev/vt/vt_core.c
  head/sys/teken/sequences
  head/sys/teken/teken.h
  head/sys/teken/teken_subr_compat.h

Modified: head/sys/dev/syscons/scterm-teken.c
==
--- head/sys/dev/syscons/scterm-teken.c Fri Aug 18 15:38:08 2017
(r322661)
+++ head/sys/dev/syscons/scterm-teken.c Fri Aug 18 15:40:40 2017
(r322662)
@@ -673,14 +673,60 @@ scteken_copy(void *arg, const teken_rect_t *r, const t
 static void
 scteken_param(void *arg, int cmd, unsigned int value)
 {
+   static int cattrs[] = {
+   0,  /* block */
+   CONS_BLINK_CURSOR,  /* blinking block */
+   CONS_CHAR_CURSOR,   /* underline */
+   CONS_CHAR_CURSOR | CONS_BLINK_CURSOR,   /* blinking underline */
+   CONS_RESET_CURSOR,  /* reset to default */
+   CONS_HIDDEN_CURSOR, /* hide cursor */
+   };
+   static int tcattrs[] = {
+   CONS_RESET_CURSOR | CONS_LOCAL_CURSOR,  /* normal */
+   CONS_HIDDEN_CURSOR | CONS_LOCAL_CURSOR, /* invisible */
+   CONS_BLINK_CURSOR | CONS_LOCAL_CURSOR,  /* very visible */
+   };
scr_stat *scp = arg;
-   int flags;
+   int flags, n, v0, v1, v2;
 
switch (cmd) {
case TP_SETBORDER:
scp->border = value & 0xff;
if (scp == scp->sc->cur_scp)
sc_set_border(scp, scp->border);
+   break;
+   case TP_SETGLOBALCURSOR:
+   n = value & 0xff;
+   v0 = (value >> 8) & 0xff;
+   v1 = (value >> 16) & 0xff;
+   v2 = (value >> 24) & 0xff;
+   switch (n) {
+   case 1: /* flags only */
+   if (v0 < sizeof(cattrs) / sizeof(cattrs[0]))
+   v0 = cattrs[v0];
+   else /* backward compatibility */
+   v0 = cattrs[v0 & 0x3];
+   sc_change_cursor_shape(scp, v0, -1, -1);
+   break;
+   case 2:
+   v2 = 0;
+   v0 &= 0x1f; /* backward compatibility */
+   v1 &= 0x1f;
+   /* FALL THROUGH */
+   case 3: /* base and height */
+   if (v2 == 0)/* count from top */
+   sc_change_cursor_shape(scp, -1,
+   scp->font_size - v1 - 1,
+   v1 - v0 + 1);
+   else if (v2 == 1) /* count from bottom */
+   sc_change_cursor_shape(scp, -1,
+   v0, v1 - v0 + 1);
+   break;
+   }
+   break;
+   case TP_SETLOCALCURSOR:
+   if (value < sizeof(tcattrs) / sizeof(tcattrs[0]))
+   sc_change_cursor_shape(scp, tcattrs[value], -1, -1);
break;
case TP_SHOWCURSOR:
if (value != 0)

Modified: head/sys/dev/vt/vt_core.c
==
--- head/sys/dev/vt/vt_core.c   Fri Aug 18 15:38:08 2017(r322661)
+++ head/sys/dev/vt/vt_core.c   Fri Aug 18 15:40:40 2017(r322662)
@@ -1052,6 +1052,15 @@ vtterm_param(struct terminal *tm, int cmd, unsigned in
struct vt_window *vw = tm->tm_softc;
 
switch (cmd) {

svn commit: r322656 - head/sys/teken

[Bruce Evans]

Author: bde
Date: Fri Aug 18 14:04:14 2017
New Revision: 322656
URL: https://svnweb.freebsd.org/changeset/base/322656

Log:
  Improve names for cons25 sequences.
  
  In a recent commit, I forgot to expand an X to an abbreviation of "BORDER".
  Fix this and some nearby bad names.
  
  The descriptions were copied from comments in scterm-sc.c, but some
  of these are bad.  The border [color] was inconsistently described as
  a property of the "display", but I had changed this to "adapter" to
  match the descriptions for other color settings.  All colors supported
  by the cons25 sequences are actually properties of the current vty and
  that should not be described.  But the other colors are defaults.
  Change "adapter" to "default" for them and remove "adapter" for the
  border.  Reduce the verbosity of the abbreviation from AD to D.

Modified:
  head/sys/teken/sequences
  head/sys/teken/teken_subr_compat.h

Modified: head/sys/teken/sequences
==
--- head/sys/teken/sequencesFri Aug 18 12:45:00 2017(r322655)
+++ head/sys/teken/sequencesFri Aug 18 14:04:14 2017(r322656)
@@ -101,11 +101,11 @@ TBC   Tab Clear   ^[ [ g  
r
 VPAVertical Position Absolute  ^[ [ d  n
 
 # Cons25 compatibility sequences
-C25ADX Cons25 set adapter border   ^[ [ = Ar
-C25ADBGCons25 set adapter background   ^[ [ = Gr
-C25ADFGCons25 set adapter foreground   ^[ [ = Fr
 C25BLPDCons25 set bell pitch duration  ^[ [ = Br r
+C25BORDCons25 set border   ^[ [ = Ar
 C25CURSCons25 set cursor type  ^[ [ = Sr
+C25DBG Cons25 set default background   ^[ [ = Gr
+C25DFG Cons25 set default foreground   ^[ [ = Fr
 C25MODECons25 set terminal mode^[ [ = Tr
 C25SGR Cons25 set graphic rendition^[ [ x  r r
 C25VTSWCons25 switch virtual terminal  ^[ [ z  r

Modified: head/sys/teken/teken_subr_compat.h
==
--- head/sys/teken/teken_subr_compat.h  Fri Aug 18 12:45:00 2017
(r322655)
+++ head/sys/teken/teken_subr_compat.h  Fri Aug 18 14:04:14 2017
(r322656)
@@ -27,7 +27,7 @@
  */
 
 static void
-teken_subr_cons25_set_adapter_border(teken_t *t, unsigned int c)
+teken_subr_cons25_set_border(teken_t *t, unsigned int c)
 {
 
teken_funcs_param(t, TP_SETBORDER, c);
@@ -44,7 +44,7 @@ static const teken_color_t cons25_colors[8] = { TC_BLA
 TC_GREEN, TC_CYAN, TC_RED, TC_MAGENTA, TC_BROWN, TC_WHITE };
 
 static void
-teken_subr_cons25_set_adapter_background(teken_t *t, unsigned int c)
+teken_subr_cons25_set_default_background(teken_t *t, unsigned int c)
 {
 
t->t_defattr.ta_bgcolor = cons25_colors[c % 8] | (c & 8);
@@ -52,7 +52,7 @@ teken_subr_cons25_set_adapter_background(teken_t *t, u
 }
 
 static void
-teken_subr_cons25_set_adapter_foreground(teken_t *t, unsigned int c)
+teken_subr_cons25_set_default_foreground(teken_t *t, unsigned int c)
 {
 
t->t_defattr.ta_fgcolor = cons25_colors[c % 8] | (c & 8);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322655 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Fri Aug 18 12:45:00 2017
New Revision: 322655
URL: https://svnweb.freebsd.org/changeset/base/322655

Log:
  Fix vt100 escape sequence for showing and hiding the cursor in syscons.
  It should toggle between 2 states, but it used a cut-down version of
  support for a related 3-state syscons escape sequence and inherited
  bugs from that.  The usual misbehaviour was that hiding and showing
  the cursor reset it to a global default.
  
  Support for the 3-state sequence remains broken by aliasing to the 2-state
  sequence.  This works better but incompatibly for the 2 cases that it
  supports.

Modified:
  head/sys/dev/syscons/scterm-teken.c

Modified: head/sys/dev/syscons/scterm-teken.c
==
--- head/sys/dev/syscons/scterm-teken.c Fri Aug 18 11:33:10 2017
(r322654)
+++ head/sys/dev/syscons/scterm-teken.c Fri Aug 18 12:45:00 2017
(r322655)
@@ -674,6 +674,7 @@ static void
 scteken_param(void *arg, int cmd, unsigned int value)
 {
scr_stat *scp = arg;
+   int flags;
 
switch (cmd) {
case TP_SETBORDER:
@@ -682,13 +683,11 @@ scteken_param(void *arg, int cmd, unsigned int value)
sc_set_border(scp, scp->border);
break;
case TP_SHOWCURSOR:
-   if (value) {
-   sc_change_cursor_shape(scp,
-   CONS_RESET_CURSOR|CONS_LOCAL_CURSOR, -1, -1);
-   } else {
-   sc_change_cursor_shape(scp,
-   CONS_HIDDEN_CURSOR|CONS_LOCAL_CURSOR, -1, -1);
-   }
+   if (value != 0)
+   flags = scp->curr_curs_attr.flags & ~CONS_HIDDEN_CURSOR;
+   else
+   flags = scp->curr_curs_attr.flags | CONS_HIDDEN_CURSOR;
+   sc_change_cursor_shape(scp, flags | CONS_LOCAL_CURSOR, -1, -1);
break;
case TP_SWITCHVT:
sc_switch_scr(scp->sc, value);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322651 - in head/sys: dev/syscons teken

[Bruce Evans]

Author: bde
Date: Fri Aug 18 10:38:49 2017
New Revision: 322651
URL: https://svnweb.freebsd.org/changeset/base/322651

Log:
  Fix missing syscons escape sequence for setting the border color.

Modified:
  head/sys/dev/syscons/scterm-teken.c
  head/sys/teken/sequences
  head/sys/teken/teken.h
  head/sys/teken/teken_subr_compat.h

Modified: head/sys/dev/syscons/scterm-teken.c
==
--- head/sys/dev/syscons/scterm-teken.c Fri Aug 18 08:05:33 2017
(r322650)
+++ head/sys/dev/syscons/scterm-teken.c Fri Aug 18 10:38:49 2017
(r322651)
@@ -676,6 +676,11 @@ scteken_param(void *arg, int cmd, unsigned int value)
scr_stat *scp = arg;
 
switch (cmd) {
+   case TP_SETBORDER:
+   scp->border = value & 0xff;
+   if (scp == scp->sc->cur_scp)
+   sc_set_border(scp, scp->border);
+   break;
case TP_SHOWCURSOR:
if (value) {
sc_change_cursor_shape(scp,

Modified: head/sys/teken/sequences
==
--- head/sys/teken/sequencesFri Aug 18 08:05:33 2017(r322650)
+++ head/sys/teken/sequencesFri Aug 18 10:38:49 2017(r322651)
@@ -101,6 +101,7 @@ TBC Tab Clear   ^[ [ g  
r
 VPAVertical Position Absolute  ^[ [ d  n
 
 # Cons25 compatibility sequences
+C25ADX Cons25 set adapter border   ^[ [ = Ar
 C25ADBGCons25 set adapter background   ^[ [ = Gr
 C25ADFGCons25 set adapter foreground   ^[ [ = Fr
 C25BLPDCons25 set bell pitch duration  ^[ [ = Br r

Modified: head/sys/teken/teken.h
==
--- head/sys/teken/teken.h  Fri Aug 18 08:05:33 2017(r322650)
+++ head/sys/teken/teken.h  Fri Aug 18 10:38:49 2017(r322651)
@@ -101,6 +101,7 @@ typedef void tf_param_t(void *, int, unsigned int);
 #defineTP_SETBELLPD_PITCH(pd)  ((pd) >> 16)
 #defineTP_SETBELLPD_DURATION(pd)   ((pd) & 0x)
 #defineTP_MOUSE6
+#defineTP_SETBORDER7
 typedef void tf_respond_t(void *, const void *, size_t);
 
 typedef struct {

Modified: head/sys/teken/teken_subr_compat.h
==
--- head/sys/teken/teken_subr_compat.h  Fri Aug 18 08:05:33 2017
(r322650)
+++ head/sys/teken/teken_subr_compat.h  Fri Aug 18 10:38:49 2017
(r322651)
@@ -27,6 +27,13 @@
  */
 
 static void
+teken_subr_cons25_set_adapter_border(teken_t *t, unsigned int c)
+{
+
+   teken_funcs_param(t, TP_SETBORDER, c);
+}
+
+static void
 teken_subr_cons25_set_cursor_type(teken_t *t, unsigned int type)
 {
 
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322634 - head/usr.sbin/vidcontrol

[Bruce Evans]

Author: bde
Date: Thu Aug 17 20:25:50 2017
New Revision: 322634
URL: https://svnweb.freebsd.org/changeset/base/322634

Log:
  I misplaced a newline in the previous commit.  Fix this, and move line
  breaks in the vt case to the same places that the man(1) puts then in
  the SYNOPSIS.  This was already done for the syscons case.  Man pages
  can't depend on the driver, and the SYNOPSIS is hard-coded for syscons
  except for -f where it is hard-coded for vt.  The usage message reverses
  this for sc.  The only other differences are whitespace and removing
  -[dL] and -l for vt.

Modified:
  head/usr.sbin/vidcontrol/vidcontrol.c

Modified: head/usr.sbin/vidcontrol/vidcontrol.c
==
--- head/usr.sbin/vidcontrol/vidcontrol.c   Thu Aug 17 20:15:20 2017
(r322633)
+++ head/usr.sbin/vidcontrol/vidcontrol.c   Thu Aug 17 20:25:50 2017
(r322634)
@@ -193,14 +193,15 @@ static void
 usage(void)
 {
if (vt4_mode)
-   fprintf(stderr, "%s\n%s\n%s\n%s\n%s\n",
+   fprintf(stderr, "%s\n%s\n%s\n%s\n%s\n%s\n",
 "usage: vidcontrol [-CHPpx] [-b color] [-c appearance] [-f [[size] file]]",
 "  [-g geometry] [-h size] [-i active | adapter | mode]",
-"  [-M char] [-m on | off] [-r foreground background]",
-"  [-S on | off] [-s number] [-T xterm | cons25] [-t N | off]",
-"  [mode] [foreground [background]] [show]");
+"  [-M char] [-m on | off]",
+"  [-r foreground background] [-S on | off] [-s number]",
+"  [-T xterm | cons25] [-t N | off] [mode]",
+"  [foreground [background]] [show]");
else
-   fprintf(stderr, "%s\n%s\n%s\n%s\n%s%s\n\n",
+   fprintf(stderr, "%s\n%s\n%s\n%s\n%s\n%s\n",
 "usage: vidcontrol [-CdHLPpx] [-b color] [-c appearance] [-f [size] file]",
 "  [-g geometry] [-h size] [-i active | adapter | mode]",
 "  [-l screen_map] [-M char] [-m on | off]",
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322626 - head/usr.sbin/vidcontrol

[Bruce Evans]

Author: bde
Date: Thu Aug 17 18:08:45 2017
New Revision: 322626
URL: https://svnweb.freebsd.org/changeset/base/322626

Log:
  Update vidcontrol -c to support all cursor appearance setting
  capabilities.
  
  Most of the capabilities (all of the arcane ones) were in FreeBSD-2.0.5,
  but were harder to use then, and vidcontrol didn't try.  FreeBSD-4.1.0
  added per-vty support, fixed the destructivness of the "destructive"
  cursor, and improved APIs.  Start using the new APIs, support all of
  their capabilities, and document all of the capabilities and some of
  the historical mistakes.
  
  vt doesn't support any of this before or after the change.
  
  Fix minor unrelated bitrot in the usage message for the syscons case.

Modified:
  head/usr.sbin/vidcontrol/vidcontrol.1
  head/usr.sbin/vidcontrol/vidcontrol.c

Modified: head/usr.sbin/vidcontrol/vidcontrol.1
==
--- head/usr.sbin/vidcontrol/vidcontrol.1   Thu Aug 17 17:17:28 2017
(r322625)
+++ head/usr.sbin/vidcontrol/vidcontrol.1   Thu Aug 17 18:08:45 2017
(r322626)
@@ -125,15 +125,123 @@ Set border color to
 This option may not be always supported by the video driver.
 .It Fl C
 Clear the history buffer.
-.It Fl c Cm normal | blink | destructive
+.It Fl c Ar setting Ns Op , Ns Ar setting ...
 Change the cursor appearance.
-The cursor is either an inverting block
-.Pq Cm normal
-that can optionally
-.Cm blink ,
-or it can be like the old hardware cursor
-.Pq Cm destructive .
-The latter is actually a simulation.
+The change is specified by a non-empty comma-separated list of
+.Cm setting Ns s .
+Each
+.Cm setting
+overrides or modifies previous ones in left to right order.
+.Pp
+The following override
+.Cm setting Ns s
+are available:
+.Bl -tag -width indent
+.It Cm block
+Set to a block covering 1 character cell,
+with a configuration-dependent coloring
+that should be at worst inverse video.
+.It Cm blinkingblock
+Set to a blinking
+.Cm block .
+.It Cm underline
+Set to
+.Cm height
+scanlines starting at
+.Cm base .
+This only gives an underline in normal configurations.
+.It Cm blinkingunderline
+Set to a blinking
+.Cm underline .
+.It Cm base Ns = Ns Ar value, Cm height Ns = Ns Ar value
+Set the specified scanline parameters.
+These parameters are only active in so-called
+.Cm char ( underline )
+mode.
+.Cm value
+is an integer in any base supported by
+.Xr strtol 3 .
+Setting
+.Cm height
+to 0 turns off the cursor in
+.Cm char
+mode.
+Negative
+.Ar value Ns s
+are silently ignored.
+Positive
+.Ar value Ns s
+are clamped to fit in the character cell when the cursor is drawn.
+.El
+.Pp
+The following override
+.Cm setting Ns s
+are provided for backwards compatibility:
+.Bl -tag -width indent
+.It Cm normal
+Old name for
+.Cm block .
+.It Cm blink
+Old name for
+.Cm blinkingblock .
+.It Cm destructive
+Bad old name for
+.Cm blinkingunderline .
+The so-called
+.Cm destructive
+cursor was only destructive due to a bug that was fixed in
+.Fx 4.1.0 .
+The destruction in the non-blinking case was so large that
+this case was not supported by
+.Nm ,
+so no backwards compatible name is needed for this case.
+.El
+.Pp
+The following modifier
+.Cm setting Ns s
+are available:
+.Bl -tag -width indent
+.It Cm blinking , noblinking
+Set or clear the blinking attribute.
+.It Cm char , nochar
+Set or clear the so-called
+.Cm char
+attribute.
+Names in this section follow the bad old names of
+flags in the implementation.
+.Cm char
+now means
+.Do
+activate the scanline parameters,
+and implement them using the best available method
+(at worst, actually use the old
+.Cm char
+method with its destruction fixed if it is configured,
+else silently ignore this flag).
+.Dc
+.It Cm hidden , nohidden
+Set or clear the hidden attribute.
+.It Cm local
+Apply the changes (except for the
+.Cm local
+and
+.Cm reset
+control flags)
+to the current vty.
+The default is to apply them to all vtys.
+.It Cm reset
+Reset everything.
+The default is to not reset.
+When the
+.Cm local
+parameter is specified, the resetting is for the 
+current local settings to the current global settings.
+Otherwise, the resetting is for the current global
+settings to defaults configured a boot time.
+Beware that the scanline parameters and not automatically
+scaled by the font size and resetting tends to undo manual
+fixups for this.
+.El
 .It Fl d
 Print out current output screen map.
 .It Xo

Modified: head/usr.sbin/vidcontrol/vidcontrol.c
==
--- head/usr.sbin/vidcontrol/vidcontrol.c   Thu Aug 17 17:17:28 2017
(r322625)
+++ head/usr.sbin/vidcontrol/vidcontrol.c   Thu Aug 17 18:08:45 2017
(r322626)
@@ -200,12 +200,13 @@ usage(void)
 "  [-S on | off] [-s number] [-T xterm | cons25] [-t N | off]",
 "  [mode] [foreground [background]] [show]");
else
-   

svn commit: r322579 - head/usr.sbin/vidcontrol

[Bruce Evans]

Author: bde
Date: Wed Aug 16 15:14:46 2017
New Revision: 322579
URL: https://svnweb.freebsd.org/changeset/base/322579

Log:
  Fix setting of the border color.  Teken doesn't support syscons' escape
  sequence "ESC [ %d A" for this although that was used here.  I will fix
  teken later, but use the more portable ioctl KDSBORDER here.
  
  The ioctl is also much easier to use if you check that it works.  For
  -b, check it and complain and exit if it failed, so that it is more
  obvious that that vt doesn't support border colors.  Don't check it
  when restoring the border color in revert(), since revert() is used
  on vt for handling other errors.
  
  Fix nearby error handling and style.  For the error of an invalid
  color, revert() and print a specific error message using err() instead
  of not revert()ing and printing spam using usage().

Modified:
  head/usr.sbin/vidcontrol/vidcontrol.c

Modified: head/usr.sbin/vidcontrol/vidcontrol.c
==
--- head/usr.sbin/vidcontrol/vidcontrol.c   Wed Aug 16 13:44:46 2017
(r322578)
+++ head/usr.sbin/vidcontrol/vidcontrol.c   Wed Aug 16 15:14:46 2017
(r322579)
@@ -154,7 +154,7 @@ revert(void)
 
ioctl(0, VT_ACTIVATE, cur_info.active_vty);
 
-   fprintf(stderr, "\033[=%dA", cur_info.console_info.mv_ovscan);
+   ioctl(0, KDSBORDER, cur_info.console_info.mv_ovscan);
fprintf(stderr, "\033[=%dH", cur_info.console_info.mv_rev.fore);
fprintf(stderr, "\033[=%dI", cur_info.console_info.mv_rev.back);
 
@@ -910,11 +910,15 @@ set_border_color(char *arg)
 {
int color;
 
-   if ((color = get_color_number(arg)) != -1) {
-   fprintf(stderr, "\033[=%dA", color);
+   color = get_color_number(arg);
+   if (color == -1) {
+   revert();
+   errx(1, "invalid color '%s'", arg);
}
-   else
-   usage();
+   if (ioctl(0, KDSBORDER, color) != 0) {
+   revert();
+   err(1, "ioctl(KD_SBORDER)");
+   }
 }
 
 static void
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322575 - in head/sys: dev/syscons sys

[Bruce Evans]

Author: bde
Date: Wed Aug 16 10:59:37 2017
New Revision: 322575
URL: https://svnweb.freebsd.org/changeset/base/322575

Log:
  Undeprecate the CONS_CURSORTYPE ioctl.  It was "deprecated" in 2001,
  but it was actually extended then and it is still used (just once) in
  /usr/src by its primary user (vidcontrol), while its replacement is
  still not used in /usr/src.
  
  yokota became inactive soon after deprecating CONS_CURSORTYPE (this
  was part of a large change to make cursor attributes per-vty).
  
  vidcontrol has incomplete support even for the old ioctl.  I will
  update it soon.  Then there are many broken escape sequences to fix.
  This is just to prepare for setting cursor colors using vidcontrol.

Modified:
  head/sys/dev/syscons/syscons.c
  head/sys/sys/consio.h

Modified: head/sys/dev/syscons/syscons.c
==
--- head/sys/dev/syscons/syscons.c  Wed Aug 16 10:38:06 2017
(r322574)
+++ head/sys/dev/syscons/syscons.c  Wed Aug 16 10:59:37 2017
(r322575)
@@ -934,7 +934,7 @@ sctty_ioctl(struct tty *tp, u_long cmd, caddr_t data, 
splx(s);
return 0;
 
-case CONS_CURSORTYPE:  /* set cursor type (obsolete) */
+case CONS_CURSORTYPE:  /* set cursor type (old interface + HIDDEN) */
s = spltty();
*(int *)data &= CONS_CURSOR_ATTRS;
sc_change_cursor_shape(scp, *(int *)data, -1, -1);

Modified: head/sys/sys/consio.h
==
--- head/sys/sys/consio.h   Wed Aug 16 10:38:06 2017(r322574)
+++ head/sys/sys/consio.h   Wed Aug 16 10:59:37 2017(r322575)
@@ -103,11 +103,15 @@ typedef struct ssaver ssaver_t;
 #define CONS_SSAVER_IOW('c', 5, ssaver_t)
 #define CONS_GSAVER_IOWR('c', 6, ssaver_t)
 
-/* set the text cursor type (obsolete, see CONS_CURSORSHAPE below) */
 /*
-#define CONS_BLINK_CURSOR (1 << 0)
-#define CONS_CHAR_CURSOR (1 << 1)
-*/
+ * Set the text cursor type.
+ *
+ * This is an old interface extended to support the CONS_HIDDEN_CURSOR bit.
+ * New code should use CONS_CURSORSHAPE.  CONS_CURSOR_ATTRS gives the 3
+ * bits supported by the (extended) old interface.  The old interface is
+ * especially unusable for hiding the cursor (even with its extension)
+ * since it changes the cursor on all vtys.
+ */
 #define CONS_CURSORTYPE_IOW('c', 7, int)
 
 /* set the bell type to audible or visual */
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322318 - in head: share/man/man4 sys/conf sys/geom

[Bruce Evans]

On Wed, 9 Aug 2017, Warner Losh wrote:


Log:
 Mark geom classes as deprecated.

 geom_bsd, geom_mbr and geom_sunlabel have been obsolete since Marcel
 Moolenaar's geom_part was in FreeBSD 7. They haven't been in GENERIC
 since FreeBSD 8. Add warning when used.

 geom_vol_ffs has been obsolete since ufs support to geom_label was
 committed in FreeBSD 5. It hasn't been in GENERIC since FreeBSD 5.
 Add warning when used.

 geom_fox has been obsolete since gmultipath was committed in FreeBSD 7.
 (no warning added, since this is a very obscure class).

 These will all be removed in FreeBSD 12.


geom_bsd (and possibly geom_mbr) can't be remove because they provide more
features than geom_part*.

The only loss that I noticed is for nested partitions on a backup drive:

GEOM_BSD + GEOM_MBR gives:

da1 da1s4aa da1s4be da1s4cs3d   da1s4e
da1s1   da1s4ac da1s4c  da1s4cs4da1s4f
da1s1a  da1s4ad da1s4cs1da1s4cs4a   da1s4g
da1s1c  da1s4b  da1s4cs2da1s4cs4c
da1s3   da1s4ba da1s4cs3da1s4cs4d
da1s4   da1s4bc da1s4cs3a   da1s4cs4e
da1s4a  da1s4bd da1s4cs3c   da1s4d

Here da1s4 = da1s4c contains an image backup of a whole drive modified
to hold a BSD label in the second sector.  The first sector contains
a copy of the MBR of the original drive, with offsets adjusted.  This
works right nested, but can only point to 4 slices.  The BSD label
points to these 4 and has 2 extras (for damaged and/or recovery areas).

The nested MBR gives slices da1s4cs[1-4].  da1s4cs[3-4] are BSD slices
containing nested labels.  These give the nested partitions
da1s4cs[3-4][a-h].

The BSD label in da1s4 gives aliases for da1s4cs[1-4].  2 of these
are for BSD slices (da1s4a == da1s4cs3 and da1s4b == da1s4cs4).  Nested
labels in these give the nested alasias da1s4[a-b][a-h].

GEOM_PART_BSD + GEOM_PART_EBR + GEM_PART_EBR_COMPAT + GEOM_PART_MBR only
gives:

da1 da1s1a  da1s4   da1s4b  da1s4e  da1s4g
da1s1   da1s3   da1s4a  da1s4d  da1s4f

It doesn't give any nesting.  It doesn't create 'c' devices for this or
any other drive.

Nesting and aliases give large complications, and I don't like even the
2-level MBR+label nesting that I normally use, but the above is what
turned up naturally and it is too hard to untangle it.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322252 - head/usr.bin/vmstat

[Bruce Evans]

On Tue, 8 Aug 2017, Emmanuel Vadot wrote:


On Tue, 8 Aug 2017 23:55:52 +1000 (EST)
Bruce Evans <b...@optusnet.com.au> wrote:


On Tue, 8 Aug 2017, Emmanuel Vadot wrote:


Log:
 vmstat: Always emit a space after the free-memory column

 When displaying in non-human form, if the free-memory number
 is large (more than 7 digits), there is no space between it and
 the page fault column.

 PR:221290
 Submitted by:  Josuah Demangeon <m...@josuah.net> (Original version)

Modified:
 head/usr.bin/vmstat/vmstat.c

Modified: head/usr.bin/vmstat/vmstat.c
==
--- head/usr.bin/vmstat/vmstat.cTue Aug  8 11:49:36 2017
(r322251)
+++ head/usr.bin/vmstat/vmstat.cTue Aug  8 12:18:11 2017
(r322252)
@@ -832,6 +832,7 @@ dovmstat(unsigned int interval, int reps)
xo_emit(" ");
xo_emit("{:free-memory/%7d}",
vmstat_pgtok(total.t_free));
+   xo_emit(" ");
}
xo_emit("{:total-page-faults/%5lu} ",
(unsigned long)rate(sum.v_vm_faults -


This seems to break the formatting.  There was a negative amount of space
available for expansion, and since the header was not expanded to match
its alignment with the fields is more random than before.  With -h, the
width was 80 columns, giving ugly line wrap on 80-column terminals with
auto-wrap.  Now it is 81 columns, giving uglier line wrap on all 80-
column terminals.


This break nothing, This was the case before too (with or without
-h), just tested in tmux with force-width 80.


Testing shows that it breaks the -H case as expected.  (I had misread the
change as affecting the -h case too.)

Before:

X procs memorypagedisks faults cpu
X r b w avm fre  flt  re  pi  pofr   sr ad0 ad1   insycs us 
sy id
X 0 0 0 12719376  977460 2167   3  28   0  2545  451   0   0  268 27162  7175  
9  1 91

Misformatting in the data line starts with the avm field taking 8 of the 7
columns reserved for it.  This and all subsequent fields are misaligned with
the header.

The header is pre-misformatted to width 85, with lots of inconsistencies in
the first line.  Old versions were carefully formatted to width 79, with no
inconsistencies in the first line.  The extra width for avm makes the extra
width for the header not even work to line up.

X procs memorypagedisks faults cpu
X r b w avm fre  flt  re  pi  pofr   sr ad0 ad1   insycs us 
sy id
X 0 0 0 12719380  977460  2167   3  28   0  2545  451   0   0  268 27162  7175  
9  1 91

The new bugs are that the extra space before the "flt" field increases the
off-by-1 error to off-by-2 for this field and all subsequent fields.  All
fields starting with "flt" now have an error of at least off-by-1 even
if none are too wide, since the header was not expanded to match.  Expansion
would increase its breakage from 6 to wide to 7 too wide.


The bugs were mostly in the first line of the header:
- the second line of the header was correct for vmstat -h
- for vmstat without -h, the second line of the header was apparently broken
   by a change like the one here, that added a space after the "r b w" fields
   without adding one in the "r b w" header
- most of the fields in the first line of the header are misaligned with the
   second lone.  Many have drifted 3 to the left of where the were in a sort
   of center-justified place.  Some of these might have actually been
   intended to be left justified, but had an off by +1 error.  Now these
   have an error of off by -2 relative to left justifications.

Only the "memory" header in the first line is better than in old versions.
It is now left justified.  Left justifying all headers in the first line
is probably best.  I couldn't find a good way to delimit the right hand
side of the extents of the headers in the first line.  The second line of
the headers already uses right justification consistently and this works
well.


I think that all this might be true but you might talk about the whole
libxo conversion that was done, not my commit right ?


I try not to look at libxo, since its misformattings are enormous starting
in the source code.

My columnize.awk script works almost perfectly to fix the above
misformattings:

X BEGIN {
X   columns = 79# XXX
X }
X 
X {

X   # Determine fields and field widths for current line.
X   # awk's field splitting feature turned out to be inadequate,
X   # and this would be even easier in C.
X   n = split($0, a, "")
X   f = 1
X   j = 1
X   while (j <= n) {
X   cpw[f] = 0
X   cfw[f] = 0
X   while (j <

Re: svn commit: r322258 - head/sys/kern

[Bruce Evans]

On Tue, 8 Aug 2017, Alan Somers wrote:


Log:
 Make p1003_1b.aio_listio_max a tunable

 p1003_1b.aio_listio_max is now a tunable. Its value is reflected in the
 sysctl of the same name, and the sysconf(3) variable _SC_AIO_LISTIO_MAX.
 Its value will be bounded from below by the compile-time constant
 AIO_LISTIO_MAX and from above by the compile-time constant
 MAX_AIO_QUEUE_PER_PROC and the tunable vfs.aio.max_aio_queue.


I don't like tunables or sysctls being limited or silently limiting them.
This is not done for more important sysctls like kern.maxfilesperproc.
Lots of silent runtime limiting is done for a few more important limits
like kern.maxfilesperproc.

The limiting is also buggy:
- vfs.aio.max_aio_queue isn't a tunable as stated.  It is a r/w sysctl
- if vfs.aio.max_aio_queue were a r/o tunable, then there would be a
  solvable ordering problem initializing it before using it
- since vfs.aio.max_aio_queue is only a sysctl, it is unusable at boot
  time (and module load time), but it is used there.

Other bugs in vfs.aio.max_aio_queue:
- its name is too long, yet not very descriptive
- its name repeats "aio"
- its name but doesn't say "len", so it looks like it limits a number of
  queues and not the length of a single queue.  This is backwards relative
  to the corresponding variable name.  That is missing "aio" when it
  is needed, but only spells "length" badly as "count".  It is
  max_queue_count, but should be something like aio_max_qlen.  All
  sysctl variables should have an aio_ prefix which is removed in the
  leaf name in the sysctl tree.
Other names have similar or worse bugs.  Bugs start with parameter names
MAX_AIO_* instead of AIO_*_MAX, and variable names track this.  There is
some ambiguity from global vs per-process limits and counts, and the
noise word "num" is sprinkled to get names which differ without describing
any difference.


Modified: head/sys/kern/vfs_aio.c
==
--- head/sys/kern/vfs_aio.c Tue Aug  8 16:06:16 2017(r322257)
+++ head/sys/kern/vfs_aio.c Tue Aug  8 16:14:31 2017(r322258)
@@ -102,6 +102,7 @@ static uint64_t jobseqno;
#endif

FEATURE(aio, "Asynchronous I/O");
+SYSCTL_DECL(_p1003_1b);

static MALLOC_DEFINE(M_LIO, "lio", "listio aio control block list");

@@ -168,6 +169,11 @@ static int max_buf_aio = MAX_BUF_AIO;
SYSCTL_INT(_vfs_aio, OID_AUTO, max_buf_aio, CTLFLAG_RW, _buf_aio, 0,
"Maximum buf aio requests per process (stored in the process)");

+static int aio_listio_max = AIO_LISTIO_MAX;
+SYSCTL_INT(_p1003_1b, CTL_P1003_1B_AIO_LISTIO_MAX, aio_listio_max,
+CTLFLAG_RDTUN | CTLFLAG_CAPRD, _listio_max, 0,
+"Maximum aio requests for a single lio_listio call");
+


The POSIX namespace is missing all of the bugs described above.  It has
aio first (except, for historical reasons it unfortunately has p10031b
instead of vfs.aio for the sysctl prefix), and max last.


#ifdef COMPAT_FREEBSD6
typedef struct oaiocb {
int aio_fildes; /* File descriptor */
@@ -388,6 +394,11 @@ static int
aio_onceonly(void)
{

+   if (aio_listio_max < AIO_LISTIO_MAX)
+   aio_listio_max = AIO_LISTIO_MAX;
+   if (aio_listio_max > MIN(MAX_AIO_QUEUE_PER_PROC, max_queue_count))
+   aio_listio_max = MIN(MAX_AIO_QUEUE_PER_PROC, max_queue_count);
+


max_queue_count is not initialized here, except to a default at compile
time.  Except for module (re)load, it always has the default value
MAX_AIO_QUEUE = 1024
  (this 1024 is bogusly ifdefed.  MAX_AIO_QUEUE is not defined in any
  header file and is not a supported option, and the tunable makes the
  option less needed than before).
MAX_AIO_QUEUE_PER_PROC = 256, modulo a similar unsupported option.
Thus MIN(MAX_AIO_QUEUE_PER_PROC, max_queue_count) is an obfuscated spelling
of 256.

MAX_AIO_QUEUE_PER_PROC is also just the default for a variable
(max_aio_queue_per_proc), so using it here has the same bugs except for
the obfuscation.

Using MIN() instead of imin() is a style bug.

OTOH, AIO_LISTIO_MAX is defined (as 16) in a header file, and is not
bogusly ifdefed.  It is 1 of 3 aio limits specified by POSIX.

The checking would be buggy if it were done later, since max_queue_count
is not checked and limiting to it can corrupt aio_listio_max (when it is
negative or just too small).

The compile-time definition of AIO_LISTIO_MAX seems to be broken.  I think
POSIX species that AIO_LISTIO_MAX shall not be defined if the value of
{AIO_LISTIO_MAX} varies at runtime, but it is still defined.  FreeBSD
has this bug for many other sysconf() variables, e.g., {OPEN_MAX}.
Perhaps AIO_LISTIO_MAX is easier to fix since it is not hard-coded as
often as OPEN_MAX.

AIO_LISTIO_MAX is now just the FreeBSD default, but the first check in
the above restricts {AIO_LISTIO_MAX} to this default (16) instead of to
the POSIX limit of _POSIX_IO_LISTIO_MAX (2).  Relaxing this check would
cause more problems with 

Re: svn commit: r322252 - head/usr.bin/vmstat

[Bruce Evans]

On Tue, 8 Aug 2017, Emmanuel Vadot wrote:


Log:
 vmstat: Always emit a space after the free-memory column

 When displaying in non-human form, if the free-memory number
 is large (more than 7 digits), there is no space between it and
 the page fault column.

 PR:221290
 Submitted by:  Josuah Demangeon  (Original version)

Modified:
 head/usr.bin/vmstat/vmstat.c

Modified: head/usr.bin/vmstat/vmstat.c
==
--- head/usr.bin/vmstat/vmstat.cTue Aug  8 11:49:36 2017
(r322251)
+++ head/usr.bin/vmstat/vmstat.cTue Aug  8 12:18:11 2017
(r322252)
@@ -832,6 +832,7 @@ dovmstat(unsigned int interval, int reps)
xo_emit(" ");
xo_emit("{:free-memory/%7d}",
vmstat_pgtok(total.t_free));
+   xo_emit(" ");
}
xo_emit("{:total-page-faults/%5lu} ",
(unsigned long)rate(sum.v_vm_faults -


This seems to break the formatting.  There was a negative amount of space
available for expansion, and since the header was not expanded to match
its alignment with the fields is more random than before.  With -h, the
width was 80 columns, giving ugly line wrap on 80-column terminals with
auto-wrap.  Now it is 81 columns, giving uglier line wrap on all 80-
column terminals.

The bugs were mostly in the first line of the header:
- the second line of the header was correct for vmstat -h
- for vmstat without -h, the second line of the header was apparently broken
  by a change like the one here, that added a space after the "r b w" fields
  without adding one in the "r b w" header
- most of the fields in the first line of the header are misaligned with the
  second lone.  Many have drifted 3 to the left of where the were in a sort
  of center-justified place.  Some of these might have actually been
  intended to be left justified, but had an off by +1 error.  Now these
  have an error of off by -2 relative to left justifications.

Only the "memory" header in the first line is better than in old versions.
It is now left justified.  Left justifying all headers in the first line
is probably best.  I couldn't find a good way to delimit the right hand
side of the extents of the headers in the first line.  The second line of
the headers already uses right justification consistently and this works
well.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321920 - head/sys/sys

[Bruce Evans]

On Sat, 5 Aug 2017, Konstantin Belousov wrote:


On Sat, Aug 05, 2017 at 12:26:03PM +1000, Bruce Evans wrote:

+#defineminor(x)((int)(x))


Another nice simplification.  Strictly, it should be (int)(dev_t)(x) since
the pseudo-prototype says that the arg is converted to dev_t, but yesterday
I couldn't see any differences even for exotic x and exotic arches.

Today I can see some difference for exotic x and perverse implementations.
E.g., x with extra bits in a large signed integer type to invoke
implementation-defined behaviour, and a perverse implementation that
truncates to 0 for direct conversion to int but does the right thing
for indirect conversion).  But we depend on the implementation doing
the right thing for other casts to int.

Also, if dev_t == uintptr_t, it is valid for the caller to keep dev_t's
in void *'s internally but not to pass void * to minor() or major()
according to the prototype.  However, casting in the macros breaks the
warning for this.  I think pure macros should not cast their args or
pretend to have prototypes, but require callers to pass args of supported
types.  The old dev_t macros were closer to this -- they have expressions
like ((x) << 8) which will fail if x is not an integral type or give
wrong results ix x has extra bits.

So you are arguing to keep the & 0x operation ?


No, I don't want to go that deep.


I think this is not needed, since your note about cast being equivalent
holds for all supported architectures and I do not see anywhere a contract
for the operations to work on non-dev_t.


The pseudo-prototypes look like they provide such a contract.  They must be
read sort of backwards as saying that it is the caller's responsibility
to pass args of the type in the pseudo-prototype, like for a K function
or a C90 function with prototype is in scope.


So are you fine with the posted patch to sys/types.h ?


OK.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r322041 - head/sys/kern

[Bruce Evans]

On Fri, 4 Aug 2017, Alan Cox wrote:


Log:
 In case readers are misled by expressions that combine multiplication and
 division, add parentheses to make the precedence explicit.

 Submitted by:  Doug Moore 
 Requested by:  imp
 Reviewed by:   imp
 MFC after: 1 week
 X-MFC after:   r321840
 Differential Revision: https://reviews.freebsd.org/D11815


This obfuscates the necessary parentheses.


Modified: head/sys/kern/subr_blist.c
...
static inline daddr_t
radix_to_skip(daddr_t radix)
{

return (radix /
-   (BLIST_BMAP_RADIX / BLIST_META_RADIX * (BLIST_META_RADIX - 1)));
+   ((BLIST_BMAP_RADIX / BLIST_META_RADIX) * (BLIST_META_RADIX - 1)));
}


Readers now have to do a more complete parse to find the interesting parts,
and writers have to count to a large number to get the count right when
the parantheses pile up at the right.

This expression is relatively simple to parse to remove the obfuscation,
but consider more complicated cases:

(1)
(a + b + c + d + e) + (f + g + h + i + j)

in floating point so that addition is not associative and the order matters.
The order is left to right in C, and this expression uses 2 sets of
parentheses to not use left to right for all terms.  Full parentheses gives
the good obfuscation:

a + b) + c) + d) + e) + f + g) + h) + i) + j)

(2) Similarly with +- instead of all +.  The order matters much more, but I
don't remember ever seeing expressions with only +- being obfuscated by
parentheses, except in floating point code where the author wants to
emphasize the left to right evaluation.  I guess there are also examples
with integer types.  Even with all + operations, the order is critical
with plain ints, since different orders might cause overflow, and with
mixed signed/unsigned/small/large integer types, the promotions depend
on the order.

(3) Similarly with */ instead of +-.  These are even more similar in
programming uses than in math structures, since + is always associative
and commutative in math structures, but it is not even commutative in
programming.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321920 - head/sys/sys

[Bruce Evans]

On Thu, 3 Aug 2017, Konstantin Belousov wrote:


On Thu, Aug 03, 2017 at 07:34:56PM +1000, Bruce Evans wrote:

I see another problem.  Masking with 0xf and casting to unsigned
are gratuitously different spellings for extracting the low 32 bits.
I prefer the cast.


Below is one more update.  I reformulated the man page text, but not too
deep.  Also I replaced masking with the cast.


OK.


diff --git a/sys/sys/types.h b/sys/sys/types.h
index 30a08724443..eacbc1ba0c4 100644
--- a/sys/sys/types.h
+++ b/sys/sys/types.h
@@ -364,9 +364,9 @@ __bitcount64(__uint64_t _x)

#include 

-#definemajor(x)((int)((dev_t)(x) >> 32)) /* major number */
-#defineminor(x)((int)((x) & 0x))   /* minor number */
-#definemakedev(x, y)   (((dev_t)(x) << 32) | (unsigned)(y)) /* create 
dev_t */
+#definemajor(x)((int)((dev_t)(x) >> 32))
+#defineminor(x)((int)(x))


Another nice simplification.  Strictly, it should be (int)(dev_t)(x) since
the pseudo-prototype says that the arg is converted to dev_t, but yesterday
I couldn't see any differences even for exotic x and exotic arches.

Today I can see some difference for exotic x and perverse implementations.
E.g., x with extra bits in a large signed integer type to invoke
implementation-defined behaviour, and a perverse implementation that
truncates to 0 for direct conversion to int but does the right thing
for indirect conversion).  But we depend on the implementation doing
the right thing for other casts to int.

Also, if dev_t == uintptr_t, it is valid for the caller to keep dev_t's
in void *'s internally but not to pass void * to minor() or major()
according to the prototype.  However, casting in the macros breaks the
warning for this.  I think pure macros should not cast their args or
pretend to have prototypes, but require callers to pass args of supported
types.  The old dev_t macros were closer to this -- they have expressions
like ((x) << 8) which will fail if x is not an integral type or give
wrong results ix x has extra bits.

POSIX had to fix related problems for the ntohl() family.  The result
seems to be that the APIs act as if they are functions.  So even if
they are implemented as macros, the macros can't just cast their args,
but must be wrapped by functions (which can be inline) to get less
forceful conversions.  FreeBSD's implementation does almost exactly
this.  It has to cast args for optimizing const args, but seems to get
this right by not casting in the non-const case.  However, I have
uncommitted "fixes" which do cast in the non-const case.  Apparently
my fixes are backwards.


+#definemakedev(x, y)   (((dev_t)(x) << 32) | (unsigned)(y))

/*
 * These declarations belong elsewhere, but are repeated here and in


Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321985 - head/sys/ofed/drivers/infiniband/core

[Bruce Evans]

On Fri, 4 Aug 2017, Colin Percival wrote:


On 08/03/17 23:28, Hans Petter Selasky wrote:

On 08/03/17 16:37, Conrad Meyer wrote:

Is it not important that the subtraction and result are evaluated
without truncation?


ticks is a circular counter. Assuming time = 0 and jiffies = -1U, then "delay"
becomes a very large negative value, because long is used, and the delay <= 0
check, is no longer working like expected.

Casting to "int" or truncating is the right thing to do in this case.


Signed integer overflow is undefined.  Using 'int' is liable to cause problems
after 2^32 ticks.


There is no (new) overflow here.  Converting a higher rank type to int has
implementation-defined behaviour.  I have yet to see an implementation that
defines its behaviour except in its source code, but all implementations
that I have seen do the right thing of truncating 2's complement integers.
Perverse implementations would produce specific garbage values or perhaps
rand().  They just have to document this.

'int overflows after 2^31-1 ticks if int is 32 bits.  It is using u_int that
is liable to cause problems after 2^32 ticks.  Using long and u_long causes
exactly the same problems if long is 32 bits.


Modified: head/sys/ofed/drivers/infiniband/core/addr.c
==
--- head/sys/ofed/drivers/infiniband/core/addr.cThu Aug  3 09:14:43
2017(r321984)
+++ head/sys/ofed/drivers/infiniband/core/addr.cThu Aug  3 09:18:25
2017(r321985)
@@ -187,10 +187,10 @@ EXPORT_SYMBOL(rdma_translate_ip);

  static void set_timeout(unsigned long time)
  {
-   unsigned long delay;
+   int delay;  /* under FreeBSD ticks are 32-bit */

 delay = time - jiffies;


There is no overflow here.  The RHS is calculated in u_long arithmetic which
has truncation instead of overflow.

First, jiffies is converted to u_long in a standards-defined way.  If
jiffies is non-negative, then its value is unchanged by the conversion.
I think jiffies can't be negative, but if it is then there are
complications -- the value must be changed; it is changed by adding a
multiple of ULONG_MAX which is normally 1, but it not easy to see that
the multiple is always 1 (if that uis the case).

Then the subtraction can't overflow, but it produces a garbage result
of time < jiffies.  As usual, using unsigned is a bug.  Here it breaks
calculations of negative differences.


-   if ((long)delay <= 0)
+   if (delay <= 0)
 delay = 1;


Then we cast to long in the old code, and assign the RHS to int in the
new code.  Both are implementation-defined.  The cast does nothing different
from assignment except it might change compiler warnings.

The old code seems to be correct enough.  Suppose time is 59 and jiffies
is 60.  Then time - jiffies is 0xLU on 32-bit systems and
0xLU on 64-bit systems.  If the implementation-defined
behaviour were documented, then we would know that the result of casting
either of these to long is -1L.  This is < 0, so delay gets fixed up to
+1.  Similarly with the new code -- the assignment converts the huge
unsigned values to -1.

Problems with the type mismatch might occur later, but I don't see any.
On 64-bit systems, the difference can be really huge without tripping
the (long)delay <= 0 test.  Above INT_MAX, later truncation to int in
APIs using ticks will give garbage (possibly 0 or negative).  I guess
that is the problem.  But even on 32-bit systems, the difference can
be INT_MAX and that seems too large to be correct.  If jiffies are
1/1000 seconds, then it is thewell-known magic number 24+ days.  User
code might generate that, but kernel driver code shouldn't.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321969 - in head/sys/boot: arm/at91/libat91 arm/ixp425/boot2 i386/boot2

[Bruce Evans]

On Thu, 3 Aug 2017, Joerg Sonnenberger wrote:


On Thu, Aug 03, 2017 at 05:53:43PM +1000, Bruce Evans wrote:

Freestanding versions (static and otherwise) cause problems with builtins.
-ffreestanding turns off all builtins.  The static memcpy used to be
ifdefed so as to use __builtin_memcpy instead of the static one if the
compiler is gcc.  That apparently broke with gcc-4.2, since the builtin
will call libc memcpy() in some cases, although memcpy() is unavailable
in the freestanding case.


GCC always had a requirement that freestanding environment must provide
certain functions (memcpy, memmove, memset, memcmp). At least memcpy and
memset are regulary used for certain code constructs. While most calls
will be inlined, it is not required.


I had forgotten about that bug.  boot2 doesn't have many fancy code
constructs, but it might have struct copying and gcc uses memcpy for
copying large structs.  clang-4.0 has the same bug.  Testing showed
1 bug nearby: for newer CPUs, it generates large code with several
movups's for small struct copies.  The bug is that -Os doesn't turn
this off.  But boot2 uses -march=i386 which does turn this off.  For
not so new CPUs like core2 where movups is slower than movaps but
movaps is unusable because the structs might not be aligned, it
normally generates large code with moves through integer registers,
but -Os works to turn this off.

The change for gcc-4.2 was apparently to support this.  The static
memcpy is not quite compatible, but is close enough and you can check
this in the generated code.  The problem is that it should be re-checked
for every change in the source code and compiler.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321920 - head/sys/sys

[Bruce Evans]

On Thu, 3 Aug 2017, Konstantin Belousov wrote:


On Thu, Aug 03, 2017 at 01:21:56PM +1000, Bruce Evans wrote:

It would be better to remove the comments.

makedev() actually has a man page (a FreeBSD addition makedev(3)).
This could have been better, and is now out of date.  The largest error
is that major() is still documented to return a value between 0 and
255.  This reduction prevented makedev() being the inverse of
major()+minor() on arbitrary args.  The man page doesn't claim that
it is, and barely gives a hint that makedev() can be used to synthesize
a dev_t from (x, y) provided 0 <= x <= 255.


See below.

diff --git a/share/man/man3/makedev.3 b/share/man/man3/makedev.3
index 87ca953dc79..a54d0590bc5 100644
--- a/share/man/man3/makedev.3
+++ b/share/man/man3/makedev.3
...
@@ -54,11 +54,18 @@ and
.Fn minor
macros can be used to obtain the original numbers from the device number


Problems with old wording:
- "can be used" is too informal.  Before this, makedev() is described as
  "allows" something.  "allow" is almost never used in share/man3/*,
  even in the sense of giving permission.  Only pthread man pages have
  many instances of this bug.
- "original" is an anachronism.  The major and minor used to be the original
  numbers, but now it is the dev_t that is original except when the dev_t
  is synthesized.  "original" might also mean "original representation".

Fixes: "can be used to obtain" is just a verbose spelling of "return".
"allows" can be improved using "returns" too, and it would be good to
shorten "a unique device number to be generated based on" to something
that says less about the one to one correspondence.  "generation" is
just a special case of applying the correspondence to get an alternative
representation.  "original" is also related to the correspondence.

Perhaps describe the correspondence first and then only say that the
functions are the ones that implement it.


.Fa dev .
+In other words, for a value
+.Va dev
+of the type
+.Vt dev_t ,
+the assertion
+.Dl dev == makedev(major(dev), minor(dev))
+is valid.


This describes the correspondence in another way.  Pehaps delete all
details about what the functions do in ther previous description and
say that they are defined by this property.  I think the inverse property
also needs to be spelled out.  It is that for (int ma, int mi)

.Dl ma = major(makedev(ma, mi))
.Dl mi = minor(makedev(ma, mi))



.Pp
In previous implementations of
.Fx
all block and character devices were uniquely identified by a pair of
-major and minor numbers.
+stable major and minor numbers.


"stable" is clarified later, but is hard to understand in context.


The major number referred to a certain device class (e.g. disks, TTYs)
while the minor number identified an instance within the device class.
Later versions of
@@ -66,7 +73,8 @@ Later versions of
automatically generate a unique device number for each character device
visible in
.Pa /dev/ .
-These numbers are not divided in device classes.
+These numbers are not divided in device classes and are not guaranteed
+to be stable upon reboot or driver reload.


They used to be stable across even more than reboot.  They were kept in
file systems, and that required them to be stable across OS versions in
practice.


.Pp
On
.Fx
@@ -78,11 +86,9 @@ conventional way.
.Sh RETURN VALUES
The
.Fn major
-macro returns a device major number that has a value between 0 and 255.
-The
+and
.Fn minor
-macro returns a device minor number whose value can span the complete
-range of an
+macros return numbers whose value can span the complete range of an
.Vt int .
.Sh SEE ALSO
.Xr mknod 2 ,
diff --git a/sys/sys/types.h b/sys/sys/types.h
index 30a08724443..8ea6e146e08 100644
--- a/sys/sys/types.h
+++ b/sys/sys/types.h
@@ -364,9 +364,9 @@ __bitcount64(__uint64_t _x)

#include 

-#definemajor(x)((int)((dev_t)(x) >> 32)) /* major number */
-#defineminor(x)((int)((x) & 0x))   /* minor number */
-#definemakedev(x, y)   (((dev_t)(x) << 32) | (unsigned)(y)) /* create 
dev_t */
+#definemajor(x)((int)((dev_t)(x) >> 32))
+#defineminor(x)((int)((x) & 0x))
+#definemakedev(x, y)   (((dev_t)(x) << 32) | (unsigned)(y))


I see another problem.  Masking with 0xf and casting to unsigned
are gratuitously different spellings for extracting the low 32 bits.
I prefer the cast.  It only works if ints have 32 bits, but we assume
that for other reasons.  The mask is especially hard to understand for
the 1's complement case.  We avoid complications with dodgy args for
major() by casting the arg to dev_t.  For minor(), the behaviour is
only clear if x is unsigned (it should be dev_t), or int or lower rank
unsigned and typeof(0x) == unsigned (true with 32-bit

Re: svn commit: r321969 - in head/sys/boot: arm/at91/libat91 arm/ixp425/boot2 i386/boot2

[Bruce Evans]

On Thu, 3 Aug 2017, Ngie Cooper wrote:


Log:
 Fix the return types for printf and putchar to match their libc and
 POSIX equivalents

 Both printf and putchar return int, not void.

 This will allow code that leverages the libcalls and checks/rely on the
 return type to interchangeably between loader code and non-loader
 code.

 MFC after: 1 month

Modified:
 head/sys/boot/arm/at91/libat91/lib.h
 head/sys/boot/arm/at91/libat91/printf.c
 head/sys/boot/arm/at91/libat91/putchar.c
 head/sys/boot/arm/ixp425/boot2/ixp425_board.c
 head/sys/boot/arm/ixp425/boot2/lib.h
 head/sys/boot/i386/boot2/boot2.c


This is wrong for at least i386/boot2.  It isn't part of the loader, and
saves space by not returning unused values.


Modified: head/sys/boot/i386/boot2/boot2.c
==
--- head/sys/boot/i386/boot2/boot2.cThu Aug  3 03:45:48 2017
(r321968)
+++ head/sys/boot/i386/boot2/boot2.cThu Aug  3 05:27:05 2017
(r321969)
@@ -114,8 +114,8 @@ void exit(int);
static void load(void);
static int parse(void);
static int dskread(void *, unsigned, unsigned);
-static void printf(const char *,...);
-static void putchar(int);
+static int printf(const char *,...);
+static int putchar(int);


These are freestanding static functions, so they have nothing to do
with library functions except their name is a hint that they are
similar.

Since they are static, -funit-at-a-time might allow the unused return values
to be optimized away.  Then returning unused values would be just an
obfuscation.

This file still has a static memcpy() which is quite different from the
libc version.  It doesn't return an unused value, and its arg types are
all different (no newfangled size_t or newerfangled restrict).

Freestanding versions (static and otherwise) cause problems with builtins.
-ffreestanding turns off all builtins.  The static memcpy used to be
ifdefed so as to use __builtin_memcpy instead of the static one if the
compiler is gcc.  That apparently broke with gcc-4.2, since the builtin
will call libc memcpy() in some cases, although memcpy() is unavailable
in the freestanding case.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321920 - head/sys/sys

[Bruce Evans]

On Wed, 2 Aug 2017, Konstantin Belousov wrote:


On Wed, Aug 02, 2017 at 02:38:50PM +0200, Hans Petter Selasky wrote:

On 08/02/17 14:36, Hans Petter Selasky wrote:

On 08/02/17 12:14, Konstantin Belousov wrote:

+#definemajor(x)((int)((dev_t)(x) >> 32))/* major number */
+#defineminor(x)((int)((x) & 0x))/* minor number */
+#definemakedev(x, y)(((dev_t)(x) << 32) | (y))/* create
dev_t */


One more comment on this issue:

I think makedev(x, y) should be declared like this, to avoid issues when
"y" is negative:

#definemakedev(x, y)(((dev_t)(x) << 32) | (unsigned int)(y))
/* create dev_t */


I think this has a lot of style bugs:
- long line constructed by blind expansion
- verbose spelling of 'unsigned' in the expansion to make the long line longer
- banal comment to make the long line.  The comment on this line is not quite
  as banal as the ones on the preceding 2 line -- those do less than echo the
  code.


And you'll probably want a final wrapping dev_t cast aswell. 128-bit
numbers are not yet there.

#define makedev(x, y)((dev_t)(((dev_t)(x) << 32) | (unsigned
int)(y)))
> /* create dev_t */


You mean 128-bit ints.


I agree with the usefulness of the y cast to unsigned type, but I am not sure
what is the use of final dev_t cast.  By the usual arithmetic conversion
rules, the final type of the '|' is the highest rank type of the operands.
Something unusual can only happen if int is wider than dev_t.


So it happens with 128 ints (or 65-bit ints) if dev_t remains 64 bits.


So I am going to commit the following update.


Almost OK.  The necessary casts and parentheses are already ugly enough.
This is the implementation, so it can assume that int is 32 bits and dev_t
is 64 bits and not have the complexity to support the general case.  Ints
are probably assumed to be 32 bits in a few thousand similar definitions
and a few million lines of code just in FreeBSD sources.


diff --git a/sys/sys/types.h b/sys/sys/types.h
index fce57e412ed..30a08724443 100644
--- a/sys/sys/types.h
+++ b/sys/sys/types.h
@@ -366,7 +366,7 @@ __bitcount64(__uint64_t _x)

#define major(x)((int)((dev_t)(x) >> 32)) /* major number */
#define minor(x)((int)((x) & 0x))   /* minor number */
-#definemakedev(x, y)   (((dev_t)(x) << 32) | (y))/* create 
dev_t */
+#definemakedev(x, y)   (((dev_t)(x) << 32) | (unsigned)(y)) /* create 
dev_t */

/*
 * These declarations belong elsewhere, but are repeated here and in


You fixed the long line by not spelling 'unsigned' verbosely and not
aligning the comments uniformly.  It would be better to remove the comments.

makedev() actually has a man page (a FreeBSD addition makedev(3)).
This could have been better, and is now out of date.  The largest error
is that major() is still documented to return a value between 0 and
255.  This reduction prevented makedev() being the inverse of
major()+minor() on arbitrary args.  The man page doesn't claim that
it is, and barely gives a hint that makedev() can be used to synthesize
a dev_t from (x, y) provided 0 <= x <= 255.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321335 - in head/sys: amd64/amd64 x86/x86

[Bruce Evans]

On Fri, 21 Jul 2017, Ryan Libby wrote:


Log:
 __pcpu: gcc -Wredundant-decls

 Pollution from counter.h made __pcpu visible in amd64/pmap.c.  Delete
 the existing extern decl of __pcpu in amd64/pmap.c and avoid referring
 to that symbol, instead accessing the pcpu region via PCPU_SET macros.
 Also delete an unused extern decl of __pcpu from mp_x86.c.


Thanks.  I use a similar quick fix.  Unfortunately, it is not complete.

clang is still broken.  It doesn't support -Wredundant-decls even to the
point of generating a warning when it is requested.

There is still a lot of pollution:
- the pollution is different on i386, so the problem doesn't occur for
  some reason, but the pollution is so convoluted that the reasons why
  it affects amd64 and doesn't affect i386 are both hard to see
- pmap.c (or any file in the same directory) doesn't include counter.h
  on either amd64 or i386.  The non-inclusion seems to be correct since
  no API in counter.h is used.  It gets this include from other pollution.

The pollution chain is very long.  On i386, machine/counter.h is
included by sys/counter.h (not polution); sys/counter.h is included
by sys/sf_buf.h (polution); sys/sf_buf.h is included by pmap.c (not
pollution).  But there is no problem on i386/pmap.c because it doesn't
declare or use __pcpu.

I didn't find the exact pollution chain on amd64, but guess it is the
same.  There is was a problem on amd64/pmap.c since it did define and
use __pcpu.


Modified: head/sys/amd64/amd64/pmap.c
==
--- head/sys/amd64/amd64/pmap.c Fri Jul 21 16:14:35 2017(r321334)
+++ head/sys/amd64/amd64/pmap.c Fri Jul 21 17:11:36 2017(r321335)
@@ -274,8 +274,6 @@ pmap_modified_bit(pmap_t pmap)
return (mask);
}

-extern struct pcpu __pcpu[];
-


My quick fix was just to remove this.  I didn't understand how this works.
Now I see that it is because counter.h is polluted by much more than the
declaration.  It includes .  That gives the complete complete
struct definition that is needed to use __pcpu.  But it is weird that
the extern variable is not declared in .  This seems to be the
result of hiding the variable too well to prevent using it in places like
counter.h, but then using it anyway.

I hoped to declare it in md_var.h, but wasn't sure that an incomplete
declaration would work there.  Now I think it does.  Any use requires
completing the direction.  In counter.h, this is done by including
.  amd64/pmap.c should have done the same.  The pollution
makes the explicit include unecessary.

The intended hiding seems to be:
- __pcpu[] is an implementation detail that should only be used for
  initialization.  Otherwise, it should be accessed using the PCPU
  access macros.  It is intentionally not declared in any header
  file.  Only the initialization code should refer to it.
- the initialization code is machdep.c, where it is declared non-extern.
So it shouldn't be declared in md_var.h either.


#if !defined(DIAGNOSTIC)
#ifdef __GNUC_GNU_INLINE__
#define PMAP_INLINE __attribute__((__gnu_inline__)) inline
@@ -1063,8 +1061,8 @@ pmap_bootstrap(vm_paddr_t *firstaddr)
kernel_pmap->pm_pcids[i].pm_pcid = PMAP_PCID_KERN;
kernel_pmap->pm_pcids[i].pm_gen = 1;
}
-   __pcpu[0].pc_pcid_next = PMAP_PCID_KERN + 1;
-   __pcpu[0].pc_pcid_gen = 1;
+   PCPU_SET(pcid_next, PMAP_PCID_KERN + 1);
+   PCPU_SET(pcid_gen, 1);
/*
 * pcpu area for APs is zeroed during AP startup.
 * pc_pcid_next and pc_pcid_gen are initialized by AP



This seems right, but I thought it was wrong at first since it is
initialization code.  On x86, the PCPU macros depend on the pcpu segment
register being initialized.  I had problems getting the order right
for implementing early use of the message buffer and printf().  Early
use of the PCPU macros finds garbage in the segment register and loads
futher garbage indirectly.  I fixed this by initializing the segment
register as the first thing done after leaving locore.

The above is in pmap_bootstrap() which is quite late so it is clear
that the segment register is initialized early enough even without my
changes.


Modified: head/sys/x86/x86/mp_x86.c
==
--- head/sys/x86/x86/mp_x86.c   Fri Jul 21 16:14:35 2017(r321334)
+++ head/sys/x86/x86/mp_x86.c   Fri Jul 21 17:11:36 2017(r321335)
@@ -90,8 +90,6 @@ int   mcount_lock;
int mp_naps;/* # of Applications processors */
int boot_cpu_id = -1;   /* designated BSP */

-extern struct pcpu __pcpu[];
-
/* AP uses this during bootstrap.  Do not staticize.  */
char *bootSTK;
int bootAP;


Perhaps the variable can be made static.  In FreeBSD-10, there is no
counter.h to abuse it, and it was only extern since it was used 

Re: svn commit: r321284 - in head/sys: amd64/include sys

[Bruce Evans]

On Fri, 21 Jul 2017, Konstantin Belousov wrote:


On Thu, Jul 20, 2017 at 04:02:02PM -0700, Ryan Libby wrote:

On Thu, Jul 20, 2017 at 3:33 AM, Konstantin Belousov
<kostik...@gmail.com> wrote:

On Thu, Jul 20, 2017 at 02:08:30AM -0700, Ryan Libby wrote:

On Thu, Jul 20, 2017 at 1:01 AM, Bruce Evans <b...@optusnet.com.au> wrote:

[...]

This bug is not very common.  There seem to be no instances of it in
 (only sys/cdefs.h uses __attribute__(()), and it seems to use
underscores for all the attributes).  Grepping sys/include/*.h for
attribute shows the following bugs:

X amd64/include/efi.h:#define   EFIABI_ATTR __attribute__((ms_abi))
X i386/include/efi.h:#defineEFIABI_ATTR /* __attribute__((ms_abi)) */ /* 
clang fails with this */
X ofed/include/rdma/ib_user_mad.h:typedef unsigned long 
__attribute__((aligned(4))) packed_ulong;
X ofed/include/rdma/ib_smi.h:} __attribute__ ((packed));
X ofed/include/rdma/ib_mad.h:} __attribute__ ((packed));
X ofed/include/rdma/ib_mad.h:} __attribute__ ((packed));

The commented-out ms_abi was only a style bug.  Now it is a larger style
bug -- it is different and worse than amd64.


I'm not sure what to do about i386 there (again beyond fixing up the
spelling in the comment).  Maybe the unsupported architectures should
just not be declaring EFIABI_ATTR at all?  (Thoughts, kib?)


I think i386 should be treated exactly same as amd64, i.e. EFIABI_ATTR
should be not defined if gcc < 4.4. Or I do not understand the scope
of the question.


It depends on whether to comment is correct.


After googling around [1] and a quick check of the spec [2], it now
seems to me that the i386 comment might just be erroneous.  I think the


Testing shows that it is still correct.  Using this attribute generates
the warning that it is ignored.


right solution for sys/i386/include/efi.h may just be to delete the
comment and leave that EFIAPI_ATTR macro definition as empty (always, no
compiler version check) in order to use the native calling convention.

[1] http://wiki.osdev.org/UEFI#Calling_Conventions
[2] http://www.uefi.org/sites/default/files/resources/UEFI_Spec_2_7.pdf


At very least, the UEFI Spec requires 16-byte alignment of the stack.
This is not guaranteed by our i386 ABI.


This UEFI pessimization is guaranteed to be not done by our i386 ABI.  We
use -mpreferred-stack-boundary=2 to prevent it for gcc, and this 4-byte
alignment is the default for clang.  This avoids the pessimization of
wasting time and space by padding the stack to a 16-byte boundary on
(non-null on 3/4 of function calls).

__aligned(16) on local variables is broken by this for gcc-4.2.1.
gcc-4.2.1 doesn't understand its own option, and doesn't generate the
necessary andl of the stack to align it in functions that need it.
gcc by default has the pessimization.

clang handles stack alignment correctly, except it doesn't support
-mpreferred-stack-boundary to control it.  It hard-codes 4-byte alignment
on i386, and does the necessary andl of the stack in functions that need
it.  Since clang on i386 also doesn't support ms_abi to change this, the
hard-coding seems to be perfect.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321284 - in head/sys: amd64/include sys

[Bruce Evans]

On Thu, 20 Jul 2017, Ryan Libby wrote:


On Thu, Jul 20, 2017 at 1:01 AM, Bruce Evans <b...@optusnet.com.au> wrote:

On Thu, 20 Jul 2017, Ryan Libby wrote:

Modified: head/sys/amd64/include/efi.h

==
--- head/sys/amd64/include/efi.hThu Jul 20 05:43:48 2017
(r321283)
+++ head/sys/amd64/include/efi.hThu Jul 20 06:47:06 2017
(r321284)
@@ -36,8 +36,14 @@
 * XXX: from gcc 6.2 manual:
 * Note, the ms_abi attribute for Microsoft Windows 64-bit targets
 * currently requires the -maccumulate-outgoing-args option.
+ *
+ * Avoid EFIABI_ATTR declarations for compilers that don't support it.
+ * GCC support began in version 4.4.
 */
+#if defined(__clang__) || defined(__GNUC__) && \
+(__GNUC__ > 4 || __GNUC__ == 4 && __GNUC_MINOR__ >= 4)
#define EFIABI_ATTR __attribute__((ms_abi))



This is still broken.  ms_abi is in the application namespace.  Thus
my hack of defining it to nothing is valid, and so is defining it to
'syntax error', but the latter shows the brokenness of EFIABI_ATTR.


To be clear, you're referring to how ms_abi is spelled here without
underscores?  I can prepare a follow-up to spell it as
__attribute__((__ms_abi__))


Yes.

...
X amd64/include/efi.h:#define   EFIABI_ATTR __attribute__((ms_abi))
X i386/include/efi.h:#defineEFIABI_ATTR /* __attribute__((ms_abi)) */ /* 
clang fails with this */
X ofed/include/rdma/ib_user_mad.h:typedef unsigned long 
__attribute__((aligned(4))) packed_ulong;
X ofed/include/rdma/ib_smi.h:} __attribute__ ((packed));
X ofed/include/rdma/ib_mad.h:} __attribute__ ((packed));
X ofed/include/rdma/ib_mad.h:} __attribute__ ((packed));

The commented-out ms_abi was only a style bug.  Now it is a larger style
bug -- it is different and worse than amd64.


I'm not sure what to do about i386 there (again beyond fixing up the
spelling in the comment).  Maybe the unsupported architectures should
just not be declaring EFIABI_ATTR at all?  (Thoughts, kib?)


Maybe it can go in x86 with a different ifdef for clang.  Or if it just
doesn't work on i386, don't define or use EFIABI_ATTR.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r321284 - in head/sys: amd64/include sys

[Bruce Evans]

On Thu, 20 Jul 2017, Ryan Libby wrote:


Log:
 efi: restrict visibility of EFIABI_ATTR-declared functions

 In-tree gcc (4.2) doesn't understand __attribute__((ms_abi))
 (EFIABI_ATTR).  Avoid declaring functions with that attribute when the
 compiler is detected to be gcc < 4.4.


Thanks.  I used to work around this using -Dms_abi=.


Modified: head/sys/amd64/include/efi.h
==
--- head/sys/amd64/include/efi.hThu Jul 20 05:43:48 2017
(r321283)
+++ head/sys/amd64/include/efi.hThu Jul 20 06:47:06 2017
(r321284)
@@ -36,8 +36,14 @@
 * XXX: from gcc 6.2 manual:
 * Note, the ms_abi attribute for Microsoft Windows 64-bit targets
 * currently requires the -maccumulate-outgoing-args option.
+ *
+ * Avoid EFIABI_ATTR declarations for compilers that don't support it.
+ * GCC support began in version 4.4.
 */
+#if defined(__clang__) || defined(__GNUC__) && \
+(__GNUC__ > 4 || __GNUC__ == 4 && __GNUC_MINOR__ >= 4)
#define EFIABI_ATTR __attribute__((ms_abi))


This is still broken.  ms_abi is in the application namespace.  Thus
my hack of defining it to nothing is valid, and so is defining it to
'syntax error', but the latter shows the brokenness of EFIABI_ATTR.


+#endif

#ifdef _KERNEL
struct uuid;

Modified: head/sys/sys/efi.h
==
--- head/sys/sys/efi.h  Thu Jul 20 05:43:48 2017(r321283)
+++ head/sys/sys/efi.h  Thu Jul 20 06:47:06 2017(r321284)
@@ -122,6 +122,9 @@ struct efi_tblhdr {
uint32_t__res;
};

+#ifdef _KERNEL
+
+#ifdef EFIABI_ATTR
struct efi_rt {
struct efi_tblhdr rt_hdr;
efi_status  (*rt_gettime)(struct efi_tm *, struct efi_tmcap *)
@@ -144,6 +147,7 @@ struct efi_rt {
efi_status  (*rt_reset)(enum efi_reset, efi_status, u_long,
efi_char *) EFIABI_ATTR;


This was more broken when it was outside of _KERNEL.


};
+#endif

struct efi_systbl {
struct efi_tblhdr st_hdr;
@@ -163,7 +167,6 @@ struct efi_systbl {
uint64_tst_cfgtbl;
};

-#ifdef _KERNEL
extern vm_paddr_t efi_systbl_phys;
#endif  /* _KERNEL */


This bug is not very common.  There seem to be no instances of it in
 (only sys/cdefs.h uses __attribute__(()), and it seems to use
underscores for all the attributes).  Grepping sys/include/*.h for
attribute shows the following bugs:

X amd64/include/efi.h:#define   EFIABI_ATTR __attribute__((ms_abi))
X i386/include/efi.h:#defineEFIABI_ATTR /* __attribute__((ms_abi)) */ /* 
clang fails with this */
X ofed/include/rdma/ib_user_mad.h:typedef unsigned long 
__attribute__((aligned(4))) packed_ulong;
X ofed/include/rdma/ib_smi.h:} __attribute__ ((packed));
X ofed/include/rdma/ib_mad.h:} __attribute__ ((packed));
X ofed/include/rdma/ib_mad.h:} __attribute__ ((packed));

The commented-out ms_abi was only a style bug.  Now it is a larger style
bug -- it is different and worse than amd64.

All uses of the packed attribute are bugs since it changes the ABI for
compilers that don't use it.  ofed has much larger bugs for it:
- it hard-codes __attribute__ instead of using __packed from cdefs.h
- it mis-hard-codes __attribute__ using the application namespace
- it uses gnu style (space before parentheses).  Perhaps ofed is supposed
  to use linux style instead of gnu style, but even linux (2.6.10 include
  directory) uses no spaces after __attribute__ more than half the time
  (it has a measly 420 instances with spaces and 538 without.  FreeBSD
  has a whole 30 instances of __attribute__ in cdefs.h and most of the
  instances not in there are bugs).

Grepping sys/*/* for attribute shows the following bugs:

X bsm/audit.h:typedef   u_int64_t   au_asflgs_t __attribute__ ((aligned 
(8)));
X libkern/explicit_bzero.c:__attribute__((weak)) void 
__explicit_bzero_hook(void *, size_t);
X libkern/explicit_bzero.c:__attribute__((weak)) void
X net/ieee8023ad_lacp.c:
__attribute__((__format__(__printf__, 2, 3)));
X net/netmap.h: uint8_t __attribute__((__aligned__(NM_CACHE_ALIGN))) sem[128];
X net/netmap_user.h:static void *__xxzt[] __attribute__ ((unused))  =
X netinet/sctp.h:#define SCTP_PACKED __attribute__((packed))
X netinet/sctp_header.h:#define SCTP_PACKED __attribute__((packed))
X netinet/sctp_input.c:__attribute__((noinline))
X netinet/sctp_input.c:__attribute__((noinline))
X netinet/sctp_os_bsd.h:#define SCTP_UNUSED __attribute__((unused))
X opencrypto/gfmult.h:#define   __aligned(x)__attribute__((__aligned__(x)))
X xen/blkif.h:  uint64_t   __attribute__((__aligned__(8))) id;
X xen/blkif.h:  uint64_t   __attribute__((__aligned__(8))) id;
X xen/xen_intr.h:   __attribute__((format(printf, 2, 3)));

Almost all can be fixed by using the FreeBSD wrapper in cdefs.h.  I think
ms_abi is MD so it doesn't belong there, but almost everything else does.

Bruce

Re: svn commit: r320901 - in head/sys: amd64/amd64 isa x86/isa

[Bruce Evans]

On Tue, 18 Jul 2017, Ian Lepore wrote:


On Tue, 2017-07-18 at 18:08 +1000, Bruce Evans wrote:



[lots of stuff about locking and accurate time-setting]

Good idea.

My inittodr() waits for the seconds register to change.? This
increases
boot time by an average of 0.5 seconds, but is needed to get a result
that is not out of date by an average of 0.5 seconds.? After the
seconds
register changes, the window for reading the other registers is
almost
1 second, but the reads must complete in a few microseconds anyway
else
the result is still too out of date.? Debugging printf()s show that
the reads usually complete in 10-50 usec with low latency (faster on
old systems with faster ISA buses).



I was considering adding a tunable to do that for people who don't mind
the extra time to boot/wakeup.??It's interesting that you label an
error of a few microseconds as "too out of date", I was thinking that
reducing the average error from .5s to .005s was good enough.

The .005s number comes from an idea of using a callout that begins as
soon as interrupts are available that checks for rtc rollover every
10ms, with the hopes that it can catch the rollover between the time
callouts start working and rootfs mount time when inittodr() gets
called.??If it can catch the rollover happening in that window, it can
latch the monotonic clock and then use that at inittodr() time to
calculate the fractional part of the time.??Even if it doesn't catch
the rollover in that time, it at least has a measurement that sets a
floor value for the fractional part, reducing the error to something
less than .5s.


Hmm, my rtcintr() already does essentially that without really trying.
It is designed to fix up the time after "suspension" in ddb breaks the
time by stopping timecounter updates.  The fixed up time is:

   (the rtc time) +
   (the last known good difference between the timecounter time and the
 rtc time) +
   (an adjustment for the drift rate)

My rtcintr() samples the difference and calibrates the drift every 64
seconds.  It uses rtc update interrupts every second to get precise
timing for the sampling and to avoid having to wait for updates to
complete before sampling.

Initially, the offset and drift are unknown, but using 0 for them would
give similar functionality to inittodr().


I have no idea yet how much time elapses between interrupts coming
online and the first inittodr() call on a typical modern x86
system.??For all my arm systems with nfsroot it's several seconds. ?If
it averages a few milliseconds on x86 systems, it isn't useful enough
to justify the extra work. ?If it's typically half a second or more it
probably is worth it.


My fastest x86 system takes 6 seconds to boot from boot2 (no loader) to
mounting root.  Surprisingly, inittodr() runs even later than mountroot
(at least on an old system).  That must be for the old feature of
defaulting the time to a timestamp on the (ffs) root partition.  Normally
that is not used, but if inittodr() ran earlier than it would have to
run again if it couldn't find a hardware rtc.  Interrupt handling starts
working much earlier than that, but I wouldn't trust all time features
to be initialized on the first call to rtcintr().

However, it is unimportant to set the time very early in the boot.  This
just makes CLOCK_REALTIME timestamps valid earlier.

The time should be fixed up similarly after real suspensions.  Then it
is even less clear what the reinitialization order actually is.


Accuracy for reading is not very important because accuracy for
writing
is very low (1-2 seconds).? My resettodr() only writes if the
difference
is more than 2 seconds.



The gross inaccuracy when writing is fixable now. ?I'm not worried
about the microsecond inaccuracies of accessing the hardware, what I
want to fix is the fact that clock_settime() may get called at x.995
and will record the time as x.0.


Very old old versions seemed to have complications related to the
hardware.  Something to do with half-second boundaries, because the
hardware is "smart" about this 0.995 error and schedules the next
update to 0.5 seconds after writing to the seconds register (this
reduces the average error to 0.5 seconds for sloppy settings)?
I can't find these in the 2004 version.


Because settime is now called from a task thread, any clock driver is
free to sleep as needed to set the clock hardware in a way that aligns
its second rollover with kernel time. ?Not efficient, but easy to do.


The syscall was always free to sleep, and it is a bug for it return before
setting the time.  Except, the syscall never claimed to even have hardware
to write to.  If there were an RTC driver, then it should support both sync
and async settings.


Even better, most rtc hardware has unused storage. ?Sometimes in the
form of battery-backed ram, sometimes in the form of things like alarm
registers that aren't used for anything and a driver can tuck away
values in them for its own u

Re: svn commit: r320901 - in head/sys: amd64/amd64 isa x86/isa

[Bruce Evans]

On Mon, 17 Jul 2017, Konstantin Belousov wrote:


On Mon, Jul 17, 2017 at 10:29:33AM -0600, Ian Lepore wrote:

I assumed the reason the efirt code was using the same mutex that
protected access to the at rtc was because under the hood, the efi rt
clock is really the same hardware too, accessed via bios code.


EFI spec states it explicitely, e.g. in the description of the GetTime()
function of the Time Services interface:

During runtime, if a PC-AT CMOS device is present in the platform the
caller must synchronize access to the device before calling GetTime().


The clock_lock mutex still protects individual cmos registers, provided
efi uses rtcin() etc., which it apparently doesn't.

The new atrtc_time_lock mutex doesn't protect even the rtc cmos
registers as a group.  Other subsystems can still access the registers
holding only their own lock or just clock_lock.  Only write accesses
would be harmful.  Read accesses do occur if rtcintr() is active, but
these only change the index register.

Even the rtc subsystem has (groups of) write accesses that are not
properly locked.  These are for (re)initializations.  Reinitializations
occur for changing the rtc rate for profiling and for resuming.  There
is no locking except possibly Giant for even the rtc_status[ab] images
vs their registers.  Giant locking might work for that, but it doesn't
work for settime() calling rtc code, at least after removing Giant
from settime().

If efi accesses the rtc directly, then that is just broken and sharing the
new mutex doesn't help much.  The new mutex only prevents contention between
efi code that uses it and rtc code that use it -- that is, just the rtc
timer parts of the rtc code (as the name of the new mutex suggests).
The following remain unprotected:
- rtc initialization code
- more seriously, rtc interrupt code and rtcin()/writertc() generally.
resettodr_mtx was no better.  But further abuse of clock_lock might have
worked.  Holding clock_lock around efi_runtime->rt_gettime() calls would
prevent both rtc timer code and rtc_intr() from contending.  I can't
find rt_gettime().  I think these sources of contention are normally
inactive if efi is used, but efi can still race with rtc initialization,
the cmos driver, and possibly acpi code.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r320901 - in head/sys: amd64/amd64 isa x86/isa

[Bruce Evans]

On Mon, 17 Jul 2017, Ian Lepore wrote:


On Thu, 2017-07-13 at 01:42 +1000, Bruce Evans wrote:

On Wed, 12 Jul 2017, Ian Lepore wrote:


Log:
  Protect access to the AT realtime clock with its own mutex.

  The mutex protecting access to the registered realtime clock should not be
  overloaded to protect access to the atrtc hardware, which might not even be
  the registered rtc. More importantly, the resettodr mutex needs to be
  eliminated to remove locking/sleeping restrictions on clock drivers, and
  that can't happen if MD code for amd64 depends on it. This change moves the
  protection into what's really being protected: access to the atrtc date and
  time registers.


The spin mutex provided some protection against timing bugs caused by
preemption, but it is impossible to hold the mutex around the user code
that is setting the time, so both the kernel and user code should check
if the operation took too long and fail/retry if it did.



I could barely read the reply, due to corruption of whitespace to spaces
and \xa0.  Formatting fixed, except all tabs are corrupted to spaces,
and there are extra blank lines after quotes.



I can't figure out what you're talking about here.  There is no
userland code involved in any of this stuff.


I seemed to have edited out examples with pseudo-code.  Times are normally
set by userland, and there is a larger race for the userland part than the
kernel part:

decide_time_to_set();
--> Preempt to run another thread.
// Back here much later.  Could be several quanta of 10 msec each
// later.  The time to set is now wrong by a large amount.
clock_settime(CLOCK_REALTIME, ...); // further races here
// Should check here to see if the above took too long, and retry
// from the top if so.  But on slow and/or loaded systems, this
// might give endless retries.  We should hold a mutex or something
// around the whole operation to try to limit preemption.

ntpd can be configured to run at high priority (-N flag, which gives
realtime priority 0 on FreeBSD), but I haven't noticed any systems
where this is used (except mine of course).  ntpd mostly uses delta-
micor-adjustments which are not affected much by the time taken to
make the adjustment.  At least old versions of ntpd don't do anything
like the above when they set or step the time using clock_settime().
They just do the syscall.  If this gives an error of only several
milliseconds, then ntpd will correct the error eventually using
delta-micro-adjustments.

The kernel could also increase the thread's priority, but never did.
Mutexes would help a bit.  Parts of the operation are now in a critical
section, which works similarly to increasing the priority to realtime 0.
But nothing in the kernel can protect agains the race in the userland
part.

Kernel code using mutexes (of any sort) should look like the above
retry loop if the mutex has to be dropped:

// Have time to set.
again:
mtx_lock_spin(_mtx);
...
r = set_time();
mtx_unlock_spin(_mtx);
if (r != 0 || operation_took_too_long())
goto again;

If set_time() has to drop the mutex, then it might abort and return
nonzero to reach the retry at the top level, or retry or continue
itself and depend on the top level retrying if necessary.


With correct code like that, spin mutexes are still good for limiting
retries.  I think it is best to try to hold one around the entire kernel
part of the operation, and release it when sleeping at lower levels.



I'm not sure what this is about either.  The current code for getting
the atrtc has no retries.


It is too incorrect to retry.


Modified: head/sys/x86/isa/atrtc.c
==
--- head/sys/x86/isa/atrtc.c  Tue Jul 11 21:55:20 2017(r320900)
+++ head/sys/x86/isa/atrtc.c  Wed Jul 12 02:42:57 2017(r320901)
@@ -53,9 +53,17 @@ __FBSDID("$FreeBSD$");
#include
#include "clock_if.h"

+/*
+ * clock_lock protects low-level access to individual hardware registers.
+ * atrtc_time_lock protects the entire sequence of accessing multiple registers
+ * to read or write the date and time.
+ */
#define   RTC_LOCKdo { if (!kdb_active) mtx_lock_spin(_lock); 
} while (0)
#define   RTC_UNLOCK  do { if (!kdb_active) 
mtx_unlock_spin(_lock); } while (0)


This has a lot of old design and implementation bugs:
- it abuses the i8254 mutex for the rtc
- it obfuscates this using macros
- locking individual register accesses is buggy.  Register accesses need to
   be locked in groups like your new code does
- the kdb_active hack is buggy (it can't use mutexes and is unnecessarily
   non-reentrant)
The locks should be separate and statically allocated.



I've always assumed the locking of individual register access was
because rtcin() and writertc() are global functions used by other parts

Re: svn commit: r320803 - head/sbin/mount

[Bruce Evans]

On Thu, 13 Jul 2017, Rodney W. Grimes wrote:


On Tuesday, July 11, 2017 02:35:15 PM Ian Lepore wrote:

I think the docs on this are pretty clear... under -u it says:

The set of options is determined by applying the options specified
in the argument to -o and finally applying the -r or -w option.

...

To shed some light on all of this I did the dig to find the original
documentaton on mount -u, which actually makes it rather clear what
and how things should be happening.  Somehow this has been lost over
the years:
csrg change to mount.c adding -u
https://svnweb.freebsd.org/csrg/sbin/mount/mount.c?r1=39329=39333

csrg change to mount.8 documenting -u
https://svnweb.freebsd.org/csrg/sbin/mount/mount.8?r1=39328=39466
This documenting of -u clearly states that /etc/fstab shall be
consulted during a mount -u, some place we lost that.


FreeBSD intentionally removed this in the change that added -o fstab
and -o current:

X Index: mount.8
X ===
X RCS file: /home/ncvs/src/sbin/mount/mount.8,v
X retrieving revision 1.25
X retrieving revision 1.26
X diff -u -1 -r1.25 -r1.26
X --- mount.8   8 Apr 1999 13:59:42 -   1.25
X +++ mount.8   7 May 1999 05:22:07 -   1.26
X @@ -32,3 +32,3 @@
X  .\" @(#)mount.8 8.8 (Berkeley) 6/16/94
X -.\" $Id: mount.8,v 1.24 1998/10/16 00:06:56 des Exp $
X +.\" $Id: mount.8,v 1.25 1999/04/08 13:59:42 ghelmer Exp $
X  .\"
X ...
X @@ -289,9 +302,5 @@
X  flag is also specified.
X -The set of options is determined by first extracting the options
X -for the file system from the
X -.Xr fstab 5
X -table,
X -then applying any options specified by the
X -.Fl o
X -argument,
X +The set of options is determined by applying the options specified 
X +in the argument to 
X +.Fl o 
X  and finally applying the


The new behaviour is easier to use.  To get the old behaviour, use
"fstab" for the first option in the list.  To not have this option,
just don't use it.  If it were forced like it used to be, then canceling
it would be difficult.  Even boolean options are only easy to cancel if
they have positive logic.  E.g., the cancelation of "ro" is "noro", but
the cancelation of "rw" is itself since "norw" gives the positive option
"ro".  "fstab" cannot be canceled by "nofstab" since that is not supported.
Non-boolean options are even more difficult to cancel.  E.g., to cancel
rsize=16384 for nfs, you have to know the current value and specify this
again.

I checked that "ro" is currently broken for -o current as well as for
-o fstab.  Both ignore the specified setting of "ro" and force "noro".
So all of:

-u # with "ro" currently set
-u -o fstab# with "ro" in fstab
-u -o current  # with "ro" currently set

change the current "ro" setting to "noro" == "ro" == "" = absense of
MNT_RDONLY in the mount flags.

This is because nmount(2) is too hard to use, and its use for "ro" is
still broken despite many attempts to fix it.  nmount(2) apparently
doesn't understand that the absence of "ro" means "noro".  So users or
utilities have to suppy "noro" to get the historical default of "rw".
mount/mount.c does this.  This is a cancelation like the ones that the
user would have to do to cancel the old forced "fstab".  The order of
the flags is important for cancelation.  mount is aware of this, but
gets it wrong anyway:

X $ svn ann mount.c | grep -C1 noro
X   1558rgrimes case 'w':
X  76198 dd options = catopt(options, "noro");
X   1558rgrimes break;
X --
X 163671 ru if ((init_flags & MNT_UPDATE) && (ro == 0))
X 163671 ru options = catopt(options, "noro");
X 186505 obrien

X $ svn log -r 163671 mount.c
X 
X r163671 | ru | 2006-10-24 22:02:29 + (Tue, 24 Oct 2006) | 12 lines
X 
X Revert rev. 1.86 by jmallett@ as it breaks "ro" mounts specified

X in /etc/fstab.
X 
X This has been happening due to the priority inversion; options

X specified on the command line should take precedence over options
X from fstab over default "noro" option, but since both the default
X "noro" and options specified on the command line (-w, -r, -o ...)
X were put into the same "options" variable, "noro" took precedence
X over fstab "ro" (this is easily visible with "mount -d").
X 
X PR:		bin/100164
X 
X 


Before r163671, the options list was initialized to "noro", so "noro"
was specified even without -u unless it is canceled by a later option.
I don't know why this didn't work -- "noro" is the correct default,
and canceling it it later should work.

r163671 moves this to the MNT_UPDATE case.  Cancelation still doesn't
work for this cases.

Manual cancelation does work: "ro" on the command line after "fstab"
or "current" works to add the "ro" that should be 

Re: svn commit: r320900 - in head/sys: fs/cd9660 fs/ext2fs fs/fifofs fs/msdosfs fs/nandfs fs/nfsclient fs/smbfs fs/tmpfs ufs/ufs

[Bruce Evans]

On Wed, 12 Jul 2017, John Baldwin wrote:


On Wednesday, July 12, 2017 08:08:42 PM Bruce Evans wrote:

On Tue, 11 Jul 2017, John Baldwin wrote:


Log:
 Consistently use vop_stdpathconf() for default pathconf values.

...
This is sort of backwards.  Not many settings are the same for most
file systems, and many that did were wrong.  Now more are wrong.


Modified: head/sys/fs/cd9660/cd9660_vnops.c
...
-   case _PC_PIPE_BUF:
-   *ap->a_retval = PIPE_BUF;
-   return (0);


PIPE_BUF isn't system wide.  cd9660 does support fifos, but setting it
for files that aren't fifos or directories is bogus.  POSIX allows such
bogus settings, and needs complicated wording to describe this.  E.g.,
PIPE_BUF for a directory is not completely bogus and it means that the
value for the directory applies to all fifos in the directory and is
ignored for all non-fifos in the directory.  It is unclear if PIPE_BUF
must be set for a directory if there is any fifo in the directory.
FreeBSD's man pages give no details about this.


PIPE_BUF is system wide because all FIFOs in FreeBSD are implemented via
a common fifo implementation that has a single size.  The wording about
PIPE_BUF from
http://pubs.opengroup.org/onlinepubs/009695399/functions/fpathconf.html:

   If path refers to a FIFO, or fildes refers to a pipe or FIFO, the
   value returned shall apply to the referenced object. If path or fildes
   refers to a directory, the value returned shall apply to any FIFO that
   exists or can be created within the directory. If path or fildes refers
   to any other type of file, it is unspecified whether an implementation
   supports an association of the variable name with the specified file.

This leaves the value for non-directories and non-FIFOs undefined.  We
could perhaps add more complexity to fail with EINVAL for types other
than VFIFO or VDIR, and filesystems that do not support FIFOs could
fail _PC_PIPE_BUF in their fs-specific vnop, but this would not be any
more correct as both behaviors fall into "undefined", but the current
one is simpler to implement.


This is what I said was unclear.  It is clear to standards lawyers but
not even hinted at in FreeBSD man pages.


This is system-wide.


case _PC_NO_TRUNC:
*ap->a_retval = 1;
return (0);


There is another bug with this, and a similar, larger one with
CHOWN_RESTRICTED, like the one for NAME_MAX.

{NAME_MAX} is variable, so defining NAME_MAX is a bug and since it
varies a lot defaulting _PC_NAME_MAX is worse than useless.

Similarly for NO_TRUNC and CHOWN_RESTRICTED, except they are POSIX
limits and always have prefixes and there are further bugs in the
defaulting:
- _POSIX_NO_TRUNC is defined as 1.  This means that all file systems
  have it.  But at least shortnames for msdosfs don't have it.  As
  previously discussed, this is not defaulted, but it is better for
  defaulting than _PC_NAME_MAX since 1 is correct for most file systems.
- _POSIX_CHOWN_RESTRICTED is defined as 1.  This means that all file
  systems have it.  Perhaps they are actually do.  It is defaulted, but
  its default value is spelled 1 instead of _POSIX_CHOWN_RESTRICTED.


- NAME_MAX.  This is very far from system-wide
- PATH_MAX.  This is system-wide.
- LINK_MAX.  This is very far from system-wide, especially after expansion
   of nlink_t.  Most file systems get this wrong.  LINK_MAX is bogusly
   defined to be the constant 32767, but the value is very variable.
   zfs_pathconf() returns INT_MAX and zfs uses something like int32_t
   internally, but zfs_getattr() clamps to LINK_MAX.  ext2fs limits to
   32000 for ext2 and 65000 for ext4 internally but ext2_pathconf()
   only returns this limit in some cases -- other cases return INT_MAX.


In theory if a filesystem supported an infinite link count or name count
it would have no filesystem-specific limit and having LINK_MAX and
NAME_MAX provide the system limits gives a reasonable default in that case.


NAME_MAX is required to give the limit for all file systems if it is
defined.  _POSIX_NAME_MAX is what gives a "reasonable" default (it actually
gives the minimum value for a "reasonable" {NAME_MAX}).

For newer POSIX limits, a macro value of 0 means that the feature might
be supported (and all its headers are present).  So it wouldn't be
such good style to use the macro value as the default -- pathconf() must
give the runtime value which is rarely 0.  We only use 0 for
_POSIX_IPV6 and _POSIX_PRIORITY_SCHEDULING which are not under pathconf().


I don't know if any filesystem supports infinite limits in practice, but
it's also true that the correct behavior in each filesystem would be to
return a value like MIN(fs_limit, system_limit), not just the native
filesystem limit.  Perhaps this could be implemented by a post-processing
step for VOP_PATHCONF that truncates values that exceed system limits to
system limits.


The file system should limit itsel

Re: svn commit: r320901 - in head/sys: amd64/amd64 isa x86/isa

[Bruce Evans]

On Wed, 12 Jul 2017, Ian Lepore wrote:


Log:
 Protect access to the AT realtime clock with its own mutex.

 The mutex protecting access to the registered realtime clock should not be
 overloaded to protect access to the atrtc hardware, which might not even be
 the registered rtc. More importantly, the resettodr mutex needs to be
 eliminated to remove locking/sleeping restrictions on clock drivers, and
 that can't happen if MD code for amd64 depends on it. This change moves the
 protection into what's really being protected: access to the atrtc date and
 time registers.


The spin mutex provided some protection against timing bugs caused by
preemption, but it is impossible to hold the mutex around the user code
that is setting the time, so both the kernel and user code should check
if the operation took too long and fail/retry if it did.

With correct code like that, spin mutexes are still good for limiting
retries.  I think it is best to try to hold one around the entire kernel
part of the operation, and release it when sleeping at lower levels.


Modified: head/sys/x86/isa/atrtc.c
==
--- head/sys/x86/isa/atrtc.cTue Jul 11 21:55:20 2017(r320900)
+++ head/sys/x86/isa/atrtc.cWed Jul 12 02:42:57 2017(r320901)
@@ -53,9 +53,17 @@ __FBSDID("$FreeBSD$");
#include 
#include "clock_if.h"

+/*
+ * clock_lock protects low-level access to individual hardware registers.
+ * atrtc_time_lock protects the entire sequence of accessing multiple registers
+ * to read or write the date and time.
+ */
#define RTC_LOCKdo { if (!kdb_active) mtx_lock_spin(_lock); } 
while (0)
#define RTC_UNLOCK  do { if (!kdb_active) mtx_unlock_spin(_lock); } 
while (0)


This has a lot of old design and implementation bugs:
- it abuses the i8254 mutex for the rtc
- it obfuscates this using macros
- locking individual register accesses is buggy.  Register accesses need to
  be locked in groups like your new code does
- the kdb_active hack is buggy (it can't use mutexes and is unnecessarily
  non-reentrant)
The locks should be separate and statically allocated.



+struct mtx atrtc_time_lock;
+MTX_SYSINIT(atrtc_lock_init, _time_lock, "atrtc", MTX_DEF);
+


I think the old mutex should be used here, after fixing it.  Locking
individual register access is a special case of locking groups of
register acceses.  It must be a spin mutex for low-level use, and that
is good for high-level use too, to complete the high-level operations
as soon as possible.

You used this mutex for efirt.c, but I think that is another error like
sharing the i8254 mutex (or the resettodr mutex).   Different clock
drivers should just use different mutexes.  Perhaps some drivers actually
need sleep mutexes.


...
@@ -352,6 +364,7 @@ atrtc_gettime(device_t dev, struct timespec *ts)
 * to make sure that no more than 240us pass after we start reading,
 * and try again if so.
 */
+   mtx_lock(_time_lock);
while (rtcin(RTC_STATUSA) & RTCSA_TUP)
continue;
critical_enter();


Note the comment about the 240us window.  On i386, this was broken by
SMPng, and almost fixed by starting the locking outside of the loop
(a spin mutex would fix it).

On i386, before SMPng, there was an splhigh() starting before the loop.
rtcin() also used splhigh().  This was only shortly before SMPng --
FreeBSD-4 had both splhigh()'s, but FreeBSD-3 had neither.

With no locking, an interrupt after the read of the status register can
occur, and interrupt handling can easily take longer than the 240us window
even if it doesn't cause preemption.

critical_enter() in the above gives poor man's locking which blocks
preemption but not-so-fast interrupt handlers.

The race is not very important since this code only runs at boot time.

The old splhigh() locking and a [spin] mutex around the loop lock out
interrupts and/or preemption for too long (0-1 seconds), but again this
is not very important since this code only runs at boot time.

This is fixed in my version, but only for spl locking.  The lock is
acquired before each test in the loop and dropped after each failing
test in the loop.  Then operation is completed as far as necessary
while the lock is still held.


@@ -369,6 +382,7 @@ atrtc_gettime(device_t dev, struct timespec *ts)
ct.year += (ct.year < 80 ? 2000 : 1900);
#endif
critical_exit();
+   mtx_unlock(_time_lock);
/* Set dow = -1 because some clocks don't set it correctly. */
ct.dow = -1;
return (clock_ct_to_ts(, ts));


The new mtx_lock() should be placed next to the old critical_enter() too.
Put these before or after the loop depending on whether races or high
latency are preferred.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to 

Re: svn commit: r320803 - head/sbin/mount

[Bruce Evans]

On Tue, 11 Jul 2017, John Baldwin wrote:


On Tuesday, July 11, 2017 02:35:15 PM Ian Lepore wrote:

I think the docs on this are pretty clear... under -u it says:

The set of options is determined by applying the options specified
in the argument to -o and finally applying the -r or -w option.

To me, that says that nothing in /etc/fstab is germane to mount -u
unless one of the other args to mount -u is -o fstab.

This change (r320803) seems like an acceptable workaround, but I think
the correct long term fix would be to not even open /etc/fstab on mount
-u without -o fstab.  But that may be harder to do than to say; I still
haven't actually looked at the code involved.


I thought that it did do that (not look in fstab, except possibly for
pathnames).


I concur with this.  I've always viewed '-u' as meaning "apply a delta
to the current configuration", not "add this delta to the fstab options
and then apply that entire set of options".   In practice, it seems that
it doesn't do either of those, but instead it seems to treats the options
passed to -o as the entire list of options.  This is perhaps a bit
surprising.


I thought that everyone knew that the behaviour is to blow away the current
options, since this is surprisingly bad.  My fingers know this, and exploit
it to reduce typing for mounting the root file system rw:
- "rw" is not an option, but is the absense of the "ro" option, so to change
  the root file system from read-only to read-write, my fingers type
  "mount /" -- this blows away "ro"
- however, all my fs's are mounted with noatime and I don't want to blow this
  away unintentionally.  My fingers know this, so they actually type
  "mount -o noatime" for the previous operation.

This is part of a large suite of design and implementation bugs.  dwmalone@
added [-u -o] "fstab" and "current" options to give more control and save
typing, but these weren't complete and they rotted with the larger design
and implementation bug nmount(2).  "current" only gives the current
settings that are in the mount flags returned by statfs(2), and "fstab"
doesn't blow away the current settings that aren't in the mount flags.
nmount(2) increased the problem by making it too easy to add unstructured
options that are not in the mount flags and moving parsing of options to
the kernel.

mount(8) with no args only displays the options in the mount flags.



For example, suppose you had this:

/etc/fstab:

/dev/md0/barufs ro,noauto,noexec0   0

# mount /bar
# mount | grep bar

ooop

/dev/md0 on /bar (ufs, local, noexec, read-only)
# mount -u -o rw /bar
/dev/md0 on /bar (ufs, local)

(We just lost "noexec" from fstab by upgrading to read-write)


Expect to lose that are in the mount flags like this.

But unstructured options tend to be sticky due to undocumented
implementation details.  This causes my most problems with nfs tcp/udp
flags and nfs read/write sizes.  Even when these were parsed in
nfs_mount(8), IIRC the absence of an option in the command line was
passed to the kernel as nothing and interpreted by the kernel as
a null change.  With nmount(2), userland does almost no option
processing so more clearly passes absent options as nothing so the
result is whatever undocumented interpretation the kernel prefers.
For read/write sizes, nfs also "fixes" wrong sizes by adjusting to
limits.  There is API for determining what sizes its ends up with,
so the only way to know what sizes you are setting is to always
set sizes that won't be adjusted and remember what these are in
userland.


# mount -u -o nosuid /bar
# mount | grep bar
/dev/md0 on /bar (ufs, local, nosuid)

(Did not lose "read-write" somehow, RO vs RW must be magically sticky?)


RW is just the default.  It is not a flag, but the absense of the "ro"
flag.

nmount(2) still handles negative options worse than mount(8) using much
much larger code, but after many years of fixes it handles this "rw"
negative option well enough.  Most generic flags have 3 variants like
"ro", "noro" and "nonoro", and "ro" also has variants "" "rw".
Non-generic flags (implemented in individual file systems) tend to
have less orthogonality.  Sometimes if the basic flag is "nofoo", there
is no flag "foo".  Sometimes there is a flag "nonofoo" to negate "foo".


# mount -u -o fstab,nosuid /bar
# mount |grep bar
/dev/md0 on /bar (ufs, local, noexec, nosuid)

(Did not honor "ro" from fstab!  Left mount RW)


Correction: after many years of fixes, nmount(2) still doesn't understand
the "ro" flag.  This example works correctly with old mount utilities
using mount(2).  This is with the same current kernel -- mount(8) from
FreeBSD-~5.2 parses the options correctly and then uses mount(2), while
mount(8) from FreeBSD-10 uses nmount(2).  Even nmount(2) can handle this,
so the bug must be in parsing or concatenating fstab options.  "fstab"
seems to work correctly with newer mount(8) for options in fstab except
"ro".


It would seem that there 

Re: svn commit: r320900 - in head/sys: fs/cd9660 fs/ext2fs fs/fifofs fs/msdosfs fs/nandfs fs/nfsclient fs/smbfs fs/tmpfs ufs/ufs

[Bruce Evans]

On Tue, 11 Jul 2017, John Baldwin wrote:


Log:
 Consistently use vop_stdpathconf() for default pathconf values.

 Update filesystems not currently using vop_stdpathconf() in pathconf
 VOPs to use vop_stdpathconf() for any configuration variables that do
 not have filesystem-specific values.  vop_stdpathconf() is used for
 variables that have system-wide settings as well as providing default
 values for some values based on system limits.  Filesystems can still
 explicitly override individual settings.


This is sort of backwards.  Not many settings are the same for most
file systems, and many that did were wrong.  Now more are wrong.


Modified: head/sys/fs/cd9660/cd9660_vnops.c
==
--- head/sys/fs/cd9660/cd9660_vnops.c   Tue Jul 11 21:52:11 2017
(r320899)
+++ head/sys/fs/cd9660/cd9660_vnops.c   Tue Jul 11 21:55:20 2017
(r320900)
@@ -792,20 +792,11 @@ cd9660_pathconf(ap)
else
*ap->a_retval = 37;
return (0);
-   case _PC_PATH_MAX:
-   *ap->a_retval = PATH_MAX;
-   return (0);


PATH_MAX is one of the few that is system-wide.


-   case _PC_PIPE_BUF:
-   *ap->a_retval = PIPE_BUF;
-   return (0);


PIPE_BUF isn't system wide.  cd9660 does support fifos, but setting it
for files that aren't fifos or directories is bogus.  POSIX allows such
bogus settings, and needs complicated wording to describe this.  E.g.,
PIPE_BUF for a directory is not completely bogus and it means that the
value for the directory applies to all fifos in the directory and is
ignored for all non-fifos in the directory.  It is unclear if PIPE_BUF
must be set for a directory if there is any fifo in the directory.
FreeBSD's man pages give no details about this.


-   case _PC_CHOWN_RESTRICTED:
-   *ap->a_retval = 1;
-   return (0);


This is system-wide.


case _PC_NO_TRUNC:
*ap->a_retval = 1;
return (0);


This is almost system-wide.


default:
-   return (EINVAL);
+   return (vop_stdpathconf(ap));


This is fail-unsafe.  It gives bogus settings like PIPE_BUF for all cases.
Case statements in file systems need to be even larger to kill defaults,
and the code becomes harder to read since it is harder to see the final
values.

vop_stdpathconf() has defaults for:
- ASYNC_IO.  Is this system-wide (all in vfs?)
- NAME_MAX.  This is very far from system-wide
- PATH_MAX.  This is system-wide.
- LINK_MAX.  This is very far from system-wide, especially after expansion
  of nlink_t.  Most file systems get this wrong.  LINK_MAX is bogusly
  defined to be the constant 32767, but the value is very variable.
  zfs_pathconf() returns INT_MAX and zfs uses something like int32_t
  internally, but zfs_getattr() clamps to LINK_MAX.  ext2fs limits to
  32000 for ext2 and 65000 for ext4 internally but ext2_pathconf()
  only returns this limit in some cases -- other cases return INT_MAX.
- MAX_CANON.  This is garbage except for terminals (and directories).
  Its value is also wrong.
- MAX_INPUT.  Like MAX_CANON.
- PIPE_BUF.  Like MAX_CANON, for fifos instead of terminals.
- CHOWN_RESTRICTED.  Wrong for file systems that don't support ownerships
  or changing them.  Might depend on mount options.
- VDISABLE.  Like MAX_CANON.

This is easy to improve:
- NAME_MAX, LINK_MAX: move to individual fs's
- MAX_CANON, MAX_INPUT, VDISABLE: move to devfs_pathconf().
- PIPE_BUF: move to fifo_pathconf()?  What about directories?


}
/* NOTREACHED */


Bogus comment.  Even lint should understand that default prevents reaching
this.


Modified: head/sys/fs/fifofs/fifo_vnops.c
==
--- head/sys/fs/fifofs/fifo_vnops.c Tue Jul 11 21:52:11 2017
(r320899)
+++ head/sys/fs/fifofs/fifo_vnops.c Tue Jul 11 21:55:20 2017
(r320900)
@@ -337,34 +336,6 @@ fifo_print(ap)
fifo_printinfo(ap->a_vp);
printf("\n");
return (0);
-}
-
-/*
- * Return POSIX pathconf information applicable to fifo's.
- */
-static int
-fifo_pathconf(ap)
-   struct vop_pathconf_args /* {
-   struct vnode *a_vp;
-   int a_name;
-   int *a_retval;
-   } */ *ap;
-{
-
-   switch (ap->a_name) {
-   case _PC_LINK_MAX:
-   *ap->a_retval = LINK_MAX;
-   return (0);
-   case _PC_PIPE_BUF:
-   *ap->a_retval = PIPE_BUF;
-   return (0);
-   case _PC_CHOWN_RESTRICTED:
-   *ap->a_retval = 1;
-   return (0);
-   default:
-   return (EINVAL);
-   }
-   /* NOTREACHED */
}


This was small and hopefully correct.  Not many settings apply to fifos.
It seems a but too small to be correct -- why would LINK_MAX apply but
not PATH_MAX?  Using the default, a 

svn commit: r320869 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Mon Jul 10 09:00:35 2017
New Revision: 320869
URL: https://svnweb.freebsd.org/changeset/base/320869

Log:
  Fix attribute flipping for cut marking in pixel mode.  The text-mode
  code was used, so the lightness bit was not flipped, so the flipping
  was unnecessarily null in some cases.  E.g., the unusal color scheme
  of lightwhite on white (white = lightgrey in kernelspeak) is not
  completely unusable, except null flipping of it gave no visible marks
  for cut marking.  Now flipping it works in pixel mode only.
  
  Fix text cursor attribute adjustment over cut marking in text mode for
  the usual cursor type (non-blinking full block).  Apply the flipping
  for cut marking first and adjust that instead of vice versa.  This
  gives a uniform color scheme for the usual text cursor type in text
  mode: a white block background with no change to the character
  foreground except for variations to avoid collisions.  The old order
  gave a white character fg with no change in the bg in non-colliding
  cases.  Versions before r316636 changed the bg to the non-cut-marked
  one about half the time using a saveunder bug; this accidentally gave
  something resembling a block cursor half the time.

Modified:
  head/sys/dev/syscons/scvgarndr.c

Modified: head/sys/dev/syscons/scvgarndr.c
==
--- head/sys/dev/syscons/scvgarndr.cMon Jul 10 08:10:45 2017
(r320868)
+++ head/sys/dev/syscons/scvgarndr.cMon Jul 10 09:00:35 2017
(r320869)
@@ -534,9 +534,9 @@ draw_txtcharcursor(scr_stat *scp, int at, u_short c, u
} else
 #endif /* SC_NO_FONT_LOADING */
{
-   a = vga_cursorattr_adj(a, TRUE);
if (flip)
a = vga_flipattr(a, TRUE);
+   a = vga_cursorattr_adj(a, TRUE);
sc_vtb_putc(>scr, at, c, a);
}
 }
@@ -903,7 +903,7 @@ vga_vgadraw_direct(scr_stat *scp, int from, int count,
a = sc_vtb_geta(>vtb, i);
 
if (flip)
-   a = vga_flipattr(a, TRUE);  /* XXX */
+   a = vga_flipattr(a, FALSE);
col1 = (a & 0x0f00) >> 8;
col2 = (a & 0xf000) >> 12;
 
@@ -957,7 +957,7 @@ vga_vgadraw_planar(scr_stat *scp, int from, int count,
for (i = from; count-- > 0; ++i) {
a = sc_vtb_geta(>vtb, i);
if (flip)
-   a = vga_flipattr(a, TRUE);  /* XXX */
+   a = vga_flipattr(a, FALSE);
col1 = a & 0x0f00;
col2 = (a & 0xf000) >> 4;
/* set background color in EGA/VGA latch */
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r320829 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Sun Jul  9 12:13:37 2017
New Revision: 320829
URL: https://svnweb.freebsd.org/changeset/base/320829

Log:
  Move open coding of construction of attributes for cut regions and
  text cursors to functions so that it is easier to fix and improve.
  This commit doesn't fix anything except for removing unnecessary
  complications and adding comments.

Modified:
  head/sys/dev/syscons/scvgarndr.c

Modified: head/sys/dev/syscons/scvgarndr.c
==
--- head/sys/dev/syscons/scvgarndr.cSun Jul  9 10:34:31 2017
(r320828)
+++ head/sys/dev/syscons/scvgarndr.cSun Jul  9 12:13:37 2017
(r320829)
@@ -344,6 +344,46 @@ static int vga_aspect_scale= 100;
 SYSCTL_INT(_machdep, OID_AUTO, vga_aspect_scale, CTLFLAG_RW,
 _aspect_scale, 0, "Aspect scale ratio (3:4):actual times 100");
 
+static u_short
+vga_flipattr(u_short a, int blink)
+{
+   if (blink)
+   a = (a & 0x8800) | ((a & 0x7000) >> 4) |
+   ((a & 0x0700) << 4);
+   else
+   a = ((a & 0xf000) >> 4) | ((a & 0x0f00) << 4);
+   return (a);
+}
+
+static u_short
+vga_cursorattr_adj(u_short a, int blink)
+{
+   /*
+* !blink means pixel mode, and the cursor attribute in that case
+* is simplistic reverse video.
+*/
+   if (!blink)
+   return (vga_flipattr(a, blink));
+
+   /*
+* The cursor attribute is usually that of the underlying char
+* with the bg changed to white.  If the bg is already white,
+* then the bg is changed to black.  The fg is usually not
+* changed, but if it is the same as the new bg then it is
+* changed to the inverse of the new bg.
+*/
+   if ((a & 0x7000) == 0x7000) {
+   a &= 0x8f00;
+   if ((a & 0x0700) == 0)
+   a |= 0x0700;
+   } else {
+   a |= 0x7000;
+   if ((a & 0x0700) == 0x0700)
+   a &= 0xf000;
+   }
+   return (a);
+}
+
 static void
 vga_setmdp(scr_stat *scp)
 {
@@ -436,8 +476,7 @@ vga_txtdraw(scr_stat *scp, int from, int count, int fl
for (p = sc_vtb_pointer(>scr, from); count-- > 0; ++from) {
c = sc_vtb_getc(>vtb, from);
a = sc_vtb_geta(>vtb, from);
-   a = (a & 0x8800) | ((a & 0x7000) >> 4) 
-   | ((a & 0x0700) << 4);
+   a = vga_flipattr(a, TRUE);
p = sc_vtb_putchar(>scr, p, c, a);
}
} else {
@@ -482,8 +521,7 @@ draw_txtcharcursor(scr_stat *scp, int at, u_short c, u
if (scp->curs_attr.base >= h)
return;
if (flip)
-   a = (a & 0x8800)
-   | ((a & 0x7000) >> 4) | ((a & 0x0700) << 4);
+   a = vga_flipattr(a, TRUE);
bcopy(font + c*h, font + sc->cursor_char*h, h);
font = font + sc->cursor_char*h;
for (i = imax(h - scp->curs_attr.base - scp->curs_attr.height, 
0);
@@ -496,18 +534,9 @@ draw_txtcharcursor(scr_stat *scp, int at, u_short c, u
} else
 #endif /* SC_NO_FONT_LOADING */
{
-   if ((a & 0x7000) == 0x7000) {
-   a &= 0x8f00;
-   if ((a & 0x0700) == 0)
-   a |= 0x0700;
-   } else {
-   a |= 0x7000;
-   if ((a & 0x0700) == 0x0700)
-   a &= 0xf000;
-   }
+   a = vga_cursorattr_adj(a, TRUE);
if (flip)
-   a = (a & 0x8800)
-   | ((a & 0x7000) >> 4) | ((a & 0x0700) << 4);
+   a = vga_flipattr(a, TRUE);
sc_vtb_putc(>scr, at, c, a);
}
 }
@@ -544,9 +573,7 @@ vga_txtcursor(scr_stat *scp, int at, int blink, int on
} else {
cursor_attr = sc_vtb_geta(>vtb, at);
if (flip)
-   cursor_attr = (cursor_attr & 0x8800)
-   | ((cursor_attr & 0x7000) >> 4)
-   | ((cursor_attr & 0x0700) << 4);
+   cursor_attr = vga_flipattr(cursor_attr, TRUE);
if (scp->status & VR_CURSOR_ON)
sc_vtb_putc(>scr, at,
sc_vtb_getc(>vtb, at),
@@ -875,13 +902,10 @@ vga_vgadraw_direct(scr_stat *scp, int from, int count,
for (i = from; count-- > 0; ++i) {
a = sc_vtb_geta(>vtb, i);
 
-   if (flip) {
-   col1 = (((a & 0x7000) >> 4) | (a & 0x0800)) >> 8;
-   col2 = (((a & 0x8000) >> 4) | (a & 0x0700)) >> 8;
- 

svn commit: r320808 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Sat Jul  8 17:30:33 2017
New Revision: 320808
URL: https://svnweb.freebsd.org/changeset/base/320808

Log:
  Add many bitmaps (now there are 13) for mouse cursors and logic to try
  to choose the best one.
  
  The old 9x13 cursor was was sort of correct for CGA 640x200 text mode,
  but distorted for all other modes.  This mode is still available on
  all systems with VGA, but stopped being useful in ~1985.  It has very
  unsquare pixels with an aspect ratio of 240:100 on 4:3 monitors.  On
  16:9 monitors, the unsquareness in this mode is reduced to only 180:100
  iff the monitor stretches the pixels to the full screen.
  
  Newer modes and systems have smaller distortions, but with many more
  variations.  Square pixels first became common with VGA 640x480 mode
  on 4:3 monitors.  However, standard VGA text mode also has 9-bit wide
  characters and only 25 lines, so it has 720x400 pixels.  This has
  unsquare pixels with an aspect ratio of 135:100 on 4:3 monitors.  On
  16:9 monitors, it gives almost-square pixels with an aspect ration of
  101:100 iff the monitor stretches, but in modes that were square on
  4:3 monitors square similar monitor stretching breaks the squareness.
  
  Guess the physical aspect ratio using heuristics.  The old version of
  X that I use is further from doing this using info from PnP monitors
  that is unavailable in syscons (X doesn't understand if the monitor
  is doing stretching and doesn't even understand how its its own mode
  changes affect the pixel size).  Monitors with aspect ratio control
  should be configured to _not_ stretch 4:3 modes to 16:9.  Otherwise,
  use the machdep.vga_aspect_scale sysctl to compensate.  Only 1 of my
  4 monitors/laptops requires this.  It always stretches to 16:9.
  
  The mouse data has new aspect ratio fields for selecting the best
  cursor and a new name field for display in debugging messages.
  
  Selecting the mouse cursor is now a slow operation so it is not done
  for every drawing of the cursor.  To avoid a new initialization method,
  it is done whenever the text cursor is set or changed.  Also remove
  dead code in settings of text cursors.
  
  Use larger mouse cursors (sometimes the full 10x16 one) for 8x8 fonts
  in cases where this works better (mostly in graphics mode).

Modified:
  head/sys/dev/syscons/scvgarndr.c
  head/sys/dev/syscons/syscons.h

Modified: head/sys/dev/syscons/scvgarndr.c
==
--- head/sys/dev/syscons/scvgarndr.cSat Jul  8 17:08:42 2017
(r320807)
+++ head/sys/dev/syscons/scvgarndr.cSat Jul  8 17:30:33 2017
(r320808)
@@ -40,6 +40,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 
 #include 
 
@@ -149,25 +150,137 @@ RENDERER_MODULE(vga, vga_set);
 struct mousedata {
u_short md_border[16];
u_short md_interior[16];
-   u_short md_width;
-   u_short md_height;
+   u_char  md_width;
+   u_char  md_height;
+   u_char  md_baspect;
+   u_char  md_iaspect;
+   const char *md_name;
 };
 
-static const struct mousedata mouse9x13 = { {
-   0xc000, 0xa000, 0x9000, 0x8800, 0x8400, 0x8200, 0x8100, 0x9780,
-   0xf200, 0x1200, 0x1900, 0x0900, 0x0f00, 0x, 0x, 0x, }, {
-   0x, 0x4000, 0x6000, 0x7000, 0x7800, 0x7c00, 0x7e00, 0x6800,
-   0x0c00, 0x0c00, 0x0600, 0x0600, 0x, 0x, 0x, 0x, },
-   9, 13,
+static const struct mousedata mouse10x16_50 = { {
+   0xC000, 0xA000, 0x9000, 0x8800, 0x8400, 0x8200, 0x8100, 0x8200,
+   0x8400, 0x8400, 0x8400, 0x9200, 0xB200, 0xA900, 0xC900, 0x8600, }, {
+   0x, 0x4000, 0x6000, 0x7000, 0x7800, 0x7C00, 0x7E00, 0x7C00,
+   0x7800, 0x7800, 0x7800, 0x6C00, 0x4C00, 0x4600, 0x0600, 0x, },
+   10, 16, 49, 52, "mouse10x16_50",
 };
 
-static const struct mousedata mouse10x16 = { {
-   0xc000, 0xa000, 0x9000, 0x8800, 0x8400, 0x8200, 0x8100, 0x8080,
-   0x8040, 0x83c0, 0x9200, 0xa900, 0xc900, 0x0480, 0x0480, 0x0300, }, {
-   0x, 0x4000, 0x6000, 0x7000, 0x7800, 0x7c00, 0x7e00, 0x7f00,
-   0x7f80, 0x7c00, 0x6c00, 0x4600, 0x0600, 0x0300, 0x0300, 0x, },
-   10, 16,
+static const struct mousedata mouse8x14_67 = { {
+   0xC000, 0xA000, 0x9000, 0x8800, 0x8400, 0x8200, 0x8100, 0x8700,
+   0x8400, 0x9200, 0xB200, 0xA900, 0xC900, 0x0600, 0x, 0x, }, {
+   0x, 0x4000, 0x6000, 0x7000, 0x7800, 0x7C00, 0x7E00, 0x7800,
+   0x7800, 0x6C00, 0x4C00, 0x4600, 0x0600, 0x, 0x, 0x, },
+   8, 14, 64, 65, "mouse8x14_67",
 };
+
+static const struct mousedata mouse8x13_75 = { {
+   0xC000, 0xA000, 0x9000, 0x8800, 0x8400, 0x8200, 0x8600, 0x8400,
+   0xB200, 0xD200, 0x0900, 0x0900, 0x0600, 0x, 0x, 0x, }, {
+   0x, 0x4000, 0x6000, 0x7000, 0x7800, 0x7C00, 0x7800, 0x7800,
+   0x4C00, 0x0C00, 0x0600, 0x0600, 0x, 0x, 0x, 0x, },
+   8, 13, 75, 80, 

svn commit: r320805 - head/sys/dev/syscons/fonts

[Bruce Evans]

Author: bde
Date: Sat Jul  8 15:01:55 2017
New Revision: 320805
URL: https://svnweb.freebsd.org/changeset/base/320805

Log:
  Add files to help manage the (vga) syscons mouse cursor.
  
  To mostly fix distortion of mouse cursors by non-square pixels, I
  needed 8 variants of the same cursor shape for large fonts and
  another 7 variants for small fonts.  Some variants are shared,
  leaving only 13 variants in 26 glyphs altogether.  Keep these in
  the BDF source file cursor.bdf.  cursor.bdf has another 5 unused
  experimental cursors in 10 glyphs.  cursor.awk is a simple awk
  script for converting this and similar bdf files into C declarations
  for copying into scvgarndr.c.  syscons doesn't use any of this yet.

Added:
  head/sys/dev/syscons/fonts/
  head/sys/dev/syscons/fonts/cursor.awk   (contents, props changed)
  head/sys/dev/syscons/fonts/cursor.bdf   (contents, props changed)

Added: head/sys/dev/syscons/fonts/cursor.awk
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/sys/dev/syscons/fonts/cursor.awk   Sat Jul  8 15:01:55 2017
(r320805)
@@ -0,0 +1,48 @@
+# $FreeBSD$
+#
+# awk script to convert a bdf file to C declarations in a form specialized
+# for the mouse cursors in syscons/scvgarndr.c.  Usage:
+# awk -f thisfile < file.bdf < file.c
+# The accompanying syscons mouse cursor bdf file has specialized comments
+# which this script converts to details in the C declarations.
+# This is not a general conversion utility, but produces reasonable output
+# if the input is for a monospaced font of size between 9x16 and 16x16.
+
+/^COMMENT cn.*mouse/ {
+   gsub("[(),]", "")
+   i = index($3, "-")
+   n = substr($3, 1, i - 1)
+   name[n] = $4
+   i = index($4, "e")
+   j = index($4, "x")
+   k = index($4, "_")
+   width[n] = substr($4, i + 1, j - i - 1)
+   height[n] = substr($4, j + 1, k - j - 1)
+   baspect[n] = $6
+   iaspect[n] = $8
+}
+state == 0 && /^STARTCHAR/ {
+   n = substr($2, 5)
+   printf("static const struct mousedata %s = { {\n\t", name[n])
+   state = 1
+}
+state >= 1 && state < 7 || state >= 7 + 16 && state < 7 + 16 + 7 {
+   state++
+   next
+}
+state >= 7 && state < 7 + 16 || state >= 7 + 16 + 7 && state < 7 + 16 + 7 +16 {
+   printf("0x%s,", $1)
+   if (state == 7 + 7 || state == 7 + 16 + 7 + 7)
+   printf("\n\t")
+   else if (state == 7 + 15)
+   printf(" }, {\n\t")
+   else if (state == 7 + 16 + 7 + 15) {
+   printf(" },\n\t%s, %s, %s, %s, \"%s\",",
+   width[n], height[n], baspect[n], iaspect[n], name[n])
+   printf("\n};\n\n")
+   state = -1
+   } else
+   printf(" ")
+   state++
+   next
+}

Added: head/sys/dev/syscons/fonts/cursor.bdf
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/sys/dev/syscons/fonts/cursor.bdf   Sat Jul  8 15:01:55 2017
(r320805)
@@ -0,0 +1,887 @@
+STARTFONT 2.1
+COMMENT
+COMMENT $FreeBSD$
+COMMENT
+COMMENT Mouse cursors for syscons.  All except some unused ones are the
+COMMENT main 10x16 one scaled down to 9x13 and/or scaled to minimize
+COMMENT distortion with non-square pixels.  Details of the scaling are
+COMMENT given in the comments.  E.g., for the main pair of glyphs cn 66-67,
+COMMENT "mouse10x16_100 9++:13 (96), 8:11 (106)" says that:
+COMMENT - the name of the glyph pair is mouse10x16_100 (it is more
+COMMENT   convenient to put the name in a comment than in STARTCHAR)
+COMMENT - the size of the glyph pair is 10x16 (this is only used as part
+COMMENT   of the name, and is also in standard bdf info)
+COMMENT - this glyph pair is designed for a pixel aspect ratio
+COMMENT   (ysize:xsize) of 100:100 (also just part of the name)
+COMMENT - the arrowhead of the border glyph is in a rectangle of 9x13
+COMMENT   pixels, but values 2 fudges larger than 9 should be tried in
+COMMENT   calculations of (diagonal) side lengths
+COMMENT - the arrowhead of the border glyph has equal side lengths iff
+COMMENT   the aspect ratio is 96:100.  Syscons makes choices based on
+COMMENT   this precise ratio for each half of the pair.
+COMMENT - 8:11 (106) gives the arrowhead rectangle and preferred aspect
+COMMENT   ratio for the interior glyph.
+COMMENT
+COMMENT cn 0-1 mouse10x16_50 7++:16 (49), 6:13 (52)
+COMMENT cn 16-17 mouse8x14_67 7++:13 (64), 6:11 (65) (360x400 0.675; also 
320x350 24:35 = 0.686 and 320x400 3:5 = 0.600)
+COMMENT cn 32-33 mouse8x13_75 6++:10 (75), 5:8 (80)
+COMMENT cn 34-35 mouse10x16_75 7++:12 (72), 6:10 (75)
+COMMENT cn 40-41 mouse9x16_84 8++:13 (78), 7:11 (82) (not used)
+COMMENT cn 48-49 mouse9x13_90 8+:12 (89), 6:9 (89)
+COMMENT cn 50-51 mouse10x16_90 10++:15 (89), 8:12 (89)
+COMMENT cn 64-65 mouse9x13_100 8:11 (106), 6:8 

Re: svn commit: r319720 - head/sys/dev/vt

[Bruce Evans]

On Sun, 11 Jun 2017, Konstantin Belousov wrote:


On Sun, Jun 11, 2017 at 07:53:47PM +1000, Bruce Evans wrote:

On Sat, 10 Jun 2017, Konstantin Belousov wrote:


On Sat, Jun 10, 2017 at 09:33:48AM -0700, Jonathan Looney wrote:

Hi Konstantin,

On Sat, Jun 10, 2017 at 2:12 AM, Konstantin Belousov <kostik...@gmail.com>
wrote:

No, acquire is only specified for loads, and release for stores.  In other
words, on some hypothetical ll/sc architecture, the atomic_add_acq()
could be implemented as follows, in asm-pseudocode
atomic_add_acq(int x):
ll  x, r1
acq x
add 1, r
sc  r1, x
Your use of the atomic does not prevent stores reordering.


If this is true, it sounds like the atomic(9) man page needs some revision.
It says:

 When an atomic operation has acquire semantics, the effects of the
 operation must have completed before any subsequent load or store (by
 program order) is performed.  Conversely, acquire semantics do not
 require that prior loads or stores have completed before the atomic
 operation is performed.

Up until now, I have relied on this description of the way acquire barriers
work. If this description is incorrect, I think it is important to fix it
quickly.


I think it is correct, but bit confusing.  Its first sentence says that
the acquire operation acts as if it completes before any subsequent load
or store operation (even non-atomic ones; even to different locations; is
it really that strong?) starts.  Its second sentence is redundant except

Yes, all operations that are sequenced before/after, in the intra-thread
sence of ordering by sequential points.


for "Conversely".  Completion of earlier operations is before the acquire
operation is not the converse of starting of later operations after the
acquire operation.

So the acquire operation gives a sort of read-write barrier that is
obviously not very useful by itself.  Using only acquire operations, I
don't see how to order anything except other acquire operations:

 acquire1
 load/store1# can be any time after acquire1
 acquire2   # must be after acquire1
 load/store2# can be any time after acquire2

Since load/store1 may be delayed until after acquire2, it may be mixed
with load/store2 in any order in general.

This is not useful at all, since it would only affect the current thread
behaviour, which is already guaranteed to be self-consistent.  You need
other thread which observes current thread behaviour, for the barriers
to have any visible effect.

And then, if there is other thread, 'sync with' is only established for
acq/rel pairs.


Yes, I forgot that the ordering requires cooperation.

So the man page is very misleading where it says "the effects of the
operation must have completed".  Only loads in the operation must have
completed.  atomic_add_acq_xxx() must also do something special with
the store for the operation to be atomic.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r319720 - head/sys/dev/vt

[Bruce Evans]

On Sat, 10 Jun 2017, Konstantin Belousov wrote:


On Sat, Jun 10, 2017 at 09:33:48AM -0700, Jonathan Looney wrote:

Hi Konstantin,

On Sat, Jun 10, 2017 at 2:12 AM, Konstantin Belousov 
wrote:

No, acquire is only specified for loads, and release for stores.  In other
words, on some hypothetical ll/sc architecture, the atomic_add_acq()
could be implemented as follows, in asm-pseudocode
atomic_add_acq(int x):
ll  x, r1
acq x
add 1, r
sc  r1, x
Your use of the atomic does not prevent stores reordering.


If this is true, it sounds like the atomic(9) man page needs some revision.
It says:

 When an atomic operation has acquire semantics, the effects of the
 operation must have completed before any subsequent load or store (by
 program order) is performed.  Conversely, acquire semantics do not
 require that prior loads or stores have completed before the atomic
 operation is performed.

Up until now, I have relied on this description of the way acquire barriers
work. If this description is incorrect, I think it is important to fix it
quickly.


I think it is correct, but bit confusing.  Its first sentence says that
the acquire operation acts as if it completes before any subsequent load
or store operation (even non-atomic ones; even to different locations; is
it really that strong?) starts.  Its second sentence is redundant except
for "Conversely".  Completion of earlier operations is before the acquire
operation is not the converse of starting of later operations after the
acquire operation.

So the acquire operation gives a sort of read-write barrier that is
obviously not very useful by itself.  Using only acquire operations, I
don't see how to order anything except other acquire operations:

acquire1
load/store1# can be any time after acquire1
acquire2   # must be after acquire1
load/store2# can be any time after acquire2

Since load/store1 may be delayed until after acquire2, it may be mixed
with load/store2 in any order in general.

Release operations give an ordering that pairs correctly with acquire
operations.  Essentially what is misdescribed as the "converse" above.


There are many issues with the atomic(9) man page, and they are not easy
to fix.  In essence, useful, implementable and rigorous specification of
the atomics behaviour or the memory model as whole (which cannot be avoided
if atomics are specified) seems to be still somewhat unsolved scientific
problem.

As is, I strongly suggest to rely on the standard C or C++
specifications and augment it with some code reading of
arch/include/atomic.h. Despite it is unfeasible to use compiler-provided
atomics in kernel in C runtime, the intended behaviour as specified in
standards give us a strong base to get something that does not have
inherent design mistakes. Unfortunately, this is the only way to get it
correct now.


But the standards are even harder to read (since they are more formal and
more general).


I'm not claiming that this fixes all bugs in this area. (In fact, I

specifically disclaim this.) But, it does stop the crash from occurring.

If you still feel there are better mechanisms to achieve the desired
ordering, please let me know and I'll be happy to fix and/or improve

this.
See the pseudocode I posted in my original reply, which uses acq/rel pair.


Try to do it without using any atomic ops directly.  Atomic ops are too
hard to use.

Well, I tried it (just reading the code).  The bug seems to be from just
foot shooting.  Normal mutex locking should work, but is not used for calls
to vt_resume_flush_timer().  I think this is not just to avoid LORs, since
initialization of callouts is done while holding the lock.


The code you posted for vt_resume_flush_timer() would switch vd_timer_armed
from 0 to 1 even if VDF_ASYNC is not set; however, that is not what we
want. vd_timer_armed should be left untouched if VDF_ASYNC is not set.

It sounds like what I want is some sort of fence in vt_upgrade() like jhb@
specified in his email. For example:

vd->vd_timer_armed = 1;
atomic_thread_fence_rel();
vd->vd_flags |= VDF_ASYNC;

I recognize a fence would be cleaner since I really only needed the barrier
and not the atomic operation. Do you agree the above would be sufficient to
ensure store ordering?


No, it is not sufficient. The release barrier on write needs a dual
acquire barrier on read to have an effect. So if you prefer to use
fence_rel(), the reader should issue fence_acq() between reading
vd_flags and doing cmpset on vd_timer_armed.


The complications seem to be mostly unnecessary since everything is under
the vt lock in vt_upgrade():

XX void
XX vt_upgrade(struct vt_device *vd)
XX {
XX  ...
XX  VT_LOCK(vd);

The lock is actually the mutex vd->vd_lock.

XX  if (vd->vd_curwindow == NULL)
XX  vd->vd_curwindow = vd->vd_windows[VT_CONSWINDOW];
XX 
XX 	register_handlers = 0;

XX  if