Re: how do I preset ddb's LINES to zero

[Andrew Cagney]

On Fri, 15 Dec 2023 at 22:59, matthew green  wrote:
>
> Andrew Cagney writes:
> > > > thanks, I'll add that (it won't help with my immediate problem of a
> > > > panic during boot though)
> > >
> > > From DDB command prompt "set $lines = 0" ...
> >
> > Um, the test framework's VM is stuck waiting for someone to hit the
> > space bar :-)
> >
> > I guess I could modify my pexpect script to do just that, but I was
> > kind of hoping I could do something like add ddb.lines=0 to the boot
> > line.
>
> try "options DB_MAX_LINE=0" in your kernel?

Right.  Unfortunately that fails the "without having to rebuild the
kernel" requirement :-)

> we have poor boot-command line support if you compare against
> say what linux can do.

booting directly to ddb is sufficient for my needs.
thanks

Re: how do I preset ddb's LINES to zero

[Andrew Cagney]

> > thanks, I'll add that (it won't help with my immediate problem of a
> > panic during boot though)
>
> From DDB command prompt "set $lines = 0" ...

Um, the test framework's VM is stuck waiting for someone to hit the
space bar :-)

I guess I could modify my pexpect script to do just that, but I was
kind of hoping I could do something like add ddb.lines=0 to the boot
line.

Re: how do I preset ddb's LINES to zero

[Andrew Cagney]

On Fri, 15 Dec 2023 at 11:22, J. Hannken-Illjes  wrote:

> > Is there a way to stop this without having to rebuild the kernel.
>
> sysctl -w ddb.lines=0

thanks, I'll add that (it won't help with my immediate problem of a
panic during boot though)

how do I preset ddb's LINES to zero

[Andrew Cagney]

Hi,

I've the stock 10.0 boot.iso booting within a KVM based test
framework.  I'd like to set things up so that should there be a panic,
it dumps registers et.al., without stopping half way waiting for
someone to hit the space bar vis:

r9  af80b451d080
r10 81d9a063
r11 0
r12 0
--db_more--

Is there a way to stop this without having to rebuild the kernel.

Andrew

Re: [PATCH] style(5): No struct typedefs

[Andrew Cagney]

On Tue, 11 Jul 2023 at 06:17, Taylor R Campbell  wrote:
>
> I propose the attached change to KNF style(5) to advise against
> typedefs for structs or unions, or pointers to them.

Perhaps be positive?  Add a section encouraging the use of struct,
union and enum forward declarations when the details aren't needed by
the header?

- I mention enums as compilers are finally getting good at checking them.
- I've found that, over time, the number of unexplainable #includes
does go down; but only when there's a willingness to go in and rip out
the #include spaghetti in .c files

> Passing around pointers to structs and unions doesn't require the
> definition, only a forward declaration:
>
> struct vnode;
> int frobnitz(struct vnode *);

Be clear about where the forward declaration should go.  Once, near
the top after the required #includes, or inline.
Also keep in mind that, just like #includes, these seem to accumulate
over time; especially when inline.

> This can dramatically cut down on header file dependencies so that
> touching one header file doesn't cause the entire kernel to rebuild.
> This also makes it clearer whether you are passing around a copy of a
> struct or a pointer to a struct, which is often semantically important
> not to conceal.
>
> Typedefs are not necessary for opaque cookies to have labels, like
> `typedef void *bus_dma_tag_t;'.  In fact, using void * here is bad
> because it prevents the C compiler from checking types; bus_dma_tag_t
> is indistinguishable from audio_dai_tag_t, and from pckbc_tag_t, and
> from acpipmtimer_t, and from sdmmc_chipset_handle_t.
>
> If we used `struct bus_dma_tag *' instead, the forward declaration
> could be `struct bus_dma_tag;' instead of having to pull in all of
> sys/bus.h, _and_ the C compiler would actually check types.  There
> isn't even any need to define `struct bus_dma_tag' anywhere; the
> pointer can be cast in sys/arch/x86/x86/bus_dma.c to `struct
> x86_bus_dma_tag', for instance (at some risk in type safety, but a
> much smaller risk than having void * as the public interface), and it
> doesn't even violate strict aliasing rules.

If a generic header were to forward declare `struct bus_dma_tag;` as
part of an abstract interface, is each implementation allowed to
provide their own local definition?  It is valid C.  The result can be
a good thing, or a dangerous thing.  Be clear either way.

(constructs such as <> were often frowned upon
because the code was crap; that's no longer true; doesn't make them a
good idea though)

I can see your point about typedef, void*, and hiding pointers, but
there are always exceptions.  For instance say the object in question
is an opaque singleton.  This week it is <> but next week it is <>.

> (Typedefs for integer, function, and function pointer types are not
> covered by this advice.)
>
> Objections?

Just get a good lock.

sadb_x_policy_reserved -> sadb_x_policy_flags

[Andrew Cagney]

I just noticed this rename when building libreswan against 10.

Can I suggest adding:
  #define sadb_x_policy_flags sadb_x_policy_flags
to signal this change to consumers?  Or perhaps there is some other
way to detect if this change is present?

As an aside, FreeBSD hid their rename by defining:
  #define sadb_x_policy_reserved sadb_x_policy_scope
from my pov that is less interesting.

Andrew

PS: the code is dumping structures

IPsec gsoc project

[Andrew Cagney]

I noticed this project on the GSOC list filed under userland:

http://wiki.netbsd.org/projects/project/ikev2/
racoon(8) is the current IKEv1 implementation used in NetBSD. The
racoon code is old and crufty and full of potential security issues.
We would like to replace it.
[Libreswan can be added to the evaluation list :-)]

Can I suggest adding a separate project focused more on updating the
IPsec kernel sources?  The work could involve gap analysis (see bug
database), refreshing the existing code, and adding missing
functionality.
While I'm not going to be much help with the code proper I can
probably help answer "why does this feature matters?"

Andrew

Re: valgrind

[Andrew Cagney]

On Fri, 25 Mar 2022 at 10:03, Reinoud Zandijk  wrote:

> > I guess they can think so because nested functions are a gcc extension.
> > Quite a useful one, I think, having been taught programming with Algol
> > 68 which of course does have nested functions; but an extension
> > nevertheless.
>
> AFAIR, it comes for GCC's support for pascal which has nested functions
> defined. Its just that for 'C' it was an easy addition since the compiler
> already suported it.

I suspect a bigger contributor may have been that all the cool
languages (and Ada) supported this feature; so why not GCC.   At the
time (pre-EGCS?) any language trying to use GCC as a front end really
struggled (except perhaps Ada), and Pascal more so than most.

However this is all hearsay.  My attempt at archology failed.  All I
found was a bug fix to the code in 1998.

Re: Request for implementation of KERN_PROC_SIGTRAMP sysctl

[Andrew Cagney]

On Sat, 16 Oct 2021 at 09:12, John Marino (NetBSD)  wrote:
>
> Alright, so I think we have established that it's impossible to
> provide KERN_PROC_SIGTRAMP as it functions on FreeBSD and DragonFly.
> The purpose of that sysctl is to get the address range of the single
> signal trampoline.

Backing up a little.  So the problem is that there's an instruction
pointer, a stack pointer, and a random set of other registers, and
we're trying to figure out the caller and its registers (aka unwind).
The way to do that is to find the unwind information.

It sounds like, for NetBSD, and a signal frame that the information
can be found in libc?

> Would it be possible to have a slightly different version on NetBSD
> that accomplishes the same thing?
> For example, call it KERN_PROC_SIGTRAMPS.  The first time it's called
> with a null oldp argument to get the result size, and the second time
> it's called with a properly sized buffer that is returned filled with
> address ranges like an array, one range for each defined signal tramp.
>
> It would be up to the program to iterate through the ranges to
> determine if a stack pointer is within a signal tramp range.
> And the gcc unwinder could be designed to only have to call it those 2
> times since the result is static per process.
>
> John

Re: ptrace(2) interface for TLSBASE

[Andrew Cagney]

On Wed, 4 Dec 2019 at 11:47, Kamil Rytarowski  wrote:
>
> Today it's missing.. do we need it in core files?

Here I'm guessing that LTSBASE is the thread-local-storage base
(virtual) register?  If that isn't included in a core file then a
debugger is going to find it, er, difficult to locate and display a
threads local-storage variables.

Even if my guess is wrong, if there's a reason for adding it to
ptrace() then most likely that same reason applies to core files.  Any
information a debugging tool can extract at runtime using either
ptrace() or /proc/PID (preferably both :-) should really be made
available in the core file.  Hence, modified memory, all registers and
the executable path (but often truncated, oops).  When the information
is missing, for instance /maps and /auxv, the debugger's left
scrambling trying to reconstruct those structures from memory (and
when the bug corrupts those structures, well ... :-)

But I digress.

> On 04.12.2019 16:56, Andrew Cagney wrote:
> > Where is it in a core file?
> >
> > On Tue, 3 Dec 2019 at 11:18, Kamil Rytarowski  wrote:
> >>
> >> TLSBASE is stored on a selection of ports in a dedicated mcontext entry,
> >> out of gpregs.
> >>
> >> In the amd64 case mcontext looks like this:
> >>
> >> typedef struct {
> >> __gregset_t __gregs;
> >> __greg_t_mc_tlsbase;
> >> __fpregset_t__fpregs;
> >> } mcontext_t;
> >>
> >> The same situation is for: i386, arm, m68k and mips.
> >>
> >> This patch implements the accessors for amd64:
> >>
> >> http://netbsd.org/~kamil/patch-00199-TLSBASE.txt
> >>
> >> Does it look correct? If so I will add the same code for i386,
> >> i386-on-amd64, arm, m68k and mips.
> >>
> >> I'm not completely sure as l_private might be desynced with TLSBASE that
> >> was updated without letting the kernel know.
> >>
> >> I intend to get this change merged into -9, before 9.0.
> >>
> >> This interface pending to be introduced in GDB/NetBSD.
> >>
> >> NB. FS/GS in x86 __gpregs is confusing as it is not TLS base.
> >>
>
>

Re: ptrace(2) interface for TLSBASE

[Andrew Cagney]

Where is it in a core file?

On Tue, 3 Dec 2019 at 11:18, Kamil Rytarowski  wrote:
>
> TLSBASE is stored on a selection of ports in a dedicated mcontext entry,
> out of gpregs.
>
> In the amd64 case mcontext looks like this:
>
> typedef struct {
> __gregset_t __gregs;
> __greg_t_mc_tlsbase;
> __fpregset_t__fpregs;
> } mcontext_t;
>
> The same situation is for: i386, arm, m68k and mips.
>
> This patch implements the accessors for amd64:
>
> http://netbsd.org/~kamil/patch-00199-TLSBASE.txt
>
> Does it look correct? If so I will add the same code for i386,
> i386-on-amd64, arm, m68k and mips.
>
> I'm not completely sure as l_private might be desynced with TLSBASE that
> was updated without letting the kernel know.
>
> I intend to get this change merged into -9, before 9.0.
>
> This interface pending to be introduced in GDB/NetBSD.
>
> NB. FS/GS in x86 __gpregs is confusing as it is not TLS base.
>

Re: __{read,write}_once

[Andrew Cagney]

On Wed, 6 Nov 2019 at 09:57, Jason Thorpe  wrote:
>
>
>
> > On Nov 6, 2019, at 5:41 AM, Kamil Rytarowski  wrote:
> >
> > On 06.11.2019 14:37, Jason Thorpe wrote:
> >>
> >>
> >>> On Nov 6, 2019, at 4:45 AM, Kamil Rytarowski  wrote:
> >>>
> >>> I propose __write_relaxed() / __read_relaxed().
> >>
> >> ...except that seems to imply the opposite of what these do.
> >>
> >> -- thorpej
> >>
> >
> > Rationale?
> >
> > This matches atomic_load_relaxed() / atomic_write_relaxed(), but we do
> > not deal with atomics here.
>
> Fair enough.  To me, the names suggest "compiler is allowed to apply relaxed 
> constraints and tear the access if it wants" But apparently the common 
> meaning is "relax, bro, I know what I'm doing".  If that's the case, I can 
> roll with it.

Honestly, without reading any code, I interpretation is more inline
with the former:
  hey relax, maybe it happens, maybe it doesn't but well, nothing
matters so what ever

Re: re-enabling debugging of 32 bit processes with 64 bit debugger

[Andrew Cagney]

On Mon, 1 Jul 2019 at 10:06, Kamil Rytarowski  wrote:
>
> On 01.07.2019 16:01, Christos Zoulas wrote:
> > In article ,
> > Michael van Elst  wrote:
> >> chris...@astron.com (Christos Zoulas) writes:
> >>
> >>> We can fill them completely with 0 as we pad with 0 the 32 bit
> >>> part of the 64 bit registers that does not exist in a 32 bit
> >>> process.
> >>
> >> The corefile pretends to be a 32bit file, so the register
> >> section should probably be similar to a coredump of a
> >> 32bit system.
> >>
> >> N.B. two registers were not zero padded, a temporary and the 'hi'
> >> register. So I don't think there is padding (or truncating)
> >> done.
> >
> > I have not looked at the core file, I was referring to what should be
> > returned by PT_GETREGS in the 64 bit tracer 32 bit tracee case.
> >
> > christos
> >
>
> From a debugger point of view, native struct reg, that is on the kernel
> side adapted (truncated registers) for 32bit tracee

Right, a 64-bit ptrace() interface must return 64-bit structures.  I'd
start with that, test it and see what breaks.

However, beyond that, the main thing is for all parties to play nice.
In general:
- when reading values the debugger is converting 64-bit raw values to
the 32-bit ISA
- when storing values the debugger should be writing to the full
64-bit ISA register
- and the kernel should kindly do the same thing
But wait, and this is where everyone gets to play nice.  There's
likely some obscure edge case where the kernel needs to leak a full
64-bit register value so that some feature works - perhaps h/w debug
registers, perhaps not.

Andrew

Re: re-enabling debugging of 32 bit processes with 64 bit debugger

[Andrew Cagney]

On Sun, 30 Jun 2019 at 10:28, Jason Thorpe  wrote:
>
>
> > On Jun 30, 2019, at 6:46 AM, Andrew Cagney  wrote:
> >
> > Things started to go down hill when the debugger developers decided
> > that rooting around in the process to extract the memory map was "a
> > good idea". After all, it too is likely been corrupted by the crashing
> > program.
> >
> > There's /proc/PID/maps, there should be a PTRACE equivalent and it
> > should also be dumped into the core file.
>
> Certainly, the ELF core files dumped by NetBSD have a PT_LOAD section for 
> each VM map entry, so the memory map information is already there.

Right (with debugger magic filling in the missing but listed read-only
segments).

>  Now, for mapped files, it doesn't record the path name like you get with 
> /proc/PID/maps, but that's something that could be fixed by having one or 
> more additional PT_NOTE sections where the note contains a PT_LOAD index to 
> file name mapping (with the path string being contained in the note).

What ever the format it might as well be consistent between  /proc,
ptrace() and core dumps, and should accommodate variable length paths
(unlike the current core file structure).  Spewing /proc/pid/maps
(actually, much of /proc/pid) as one or more notes into core file is a
good starting point (and one known to work).   Other formats are also
valid.

(I thought 9 had forked)

> -- thorpej
>

Re: re-enabling debugging of 32 bit processes with 64 bit debugger

[Andrew Cagney]

On Sat, 29 Jun 2019 at 16:04, Christos Zoulas  wrote:
>
> In article 
> ,
> Andrew Cagney   wrote:
> >
> >Having 32-bit and 64-bit debuggers isn't sufficient.  Specifically, it
> >can't handle an exec() call where the new executable has a different
> >ISA; and this imnsho is a must have.
>
> It is really hard to make a 32 bit debugger work on a 64 bit system
> because of the tricks we play with the location of the shared
> libraries in rtld and the debugger needs to be aware of them.

Yes and no.

Things started to go down hill when the debugger developers decided
that rooting around in the process to extract the memory map was "a
good idea". After all, it too is likely been corrupted by the crashing
program.

There's /proc/PID/maps, there should be a PTRACE equivalent and it
should also be dumped into the core file.


> In retrospect it would have been simpler (and uglier) to have
> /32 and /64 in all the shared library paths so that they would
> not occupy the filesystem space, but even then things could break
> for raw dlopen() calls, or opening other data files that are not
> size neutral. HP/UX with Context Dependent Files and IBM/AIX with
> Hidden Directories were attempts to a solution, but they created
> a new dimension of problems.
>
> christos
>

Re: re-enabling debugging of 32 bit processes with 64 bit debugger

[Andrew Cagney]

> > What do you think? SHould we make the code work like before? Or this is
> > functionality that we don't want to have because it is "dumb"? (I think
> > that Max here means that it adds complexity and it could be dangerous,
> > but I am just guessing)
>
> I'm baffled that this is even debatable.  The system supports running 32 bit 
> code in a 64 bit system.  Obviously you must be able to debug such processes.
>
> I suppose you could claim it would suffice to build two debuggers, one for 
> each target.  But that makes no sense.  All the toolchains are 
> multi-architecture: you can compile for 32 or 64 bit at the drop of a switch, 
> and you can link all that with a single toolchain. GDB has supported 
> multi-arch for a long time (in fact, not just multiple width but entirely 
> different ISAs from a single image).  So it would be thoroughly strange to 
> say that this sort of basic flexibility and user-friendliness is to be 
> abandoned here.  And why would NetBSD want to regress like that?  Other 
> platforms do this as a matter of course; it seems odd for NetBSD even to 
> consider looking technically inferior in this respect.

Having 32-bit and 64-bit debuggers isn't sufficient.  Specifically, it
can't handle an exec() call where the new executable has a different
ISA; and this imnsho is a must have.

Andrew

Re: RFC: New userspace fetch/store API

[Andrew Cagney]

On Mon, 25 Feb 2019 at 11:35, Eduardo Horvath  wrote:
>
> On Sat, 23 Feb 2019, Jason Thorpe wrote:
>
> > int ufetch_8(const uint8_t *uaddr, uint8_t *valp);
> > int ufetch_16(const uint16_t *uaddr, uint16_t *valp);
> > int ufetch_32(const uint32_t *uaddr, uint32_t *valp);
> > #ifdef _LP64
> > int ufetch_64(const uint64_t *uaddr, uint64_t *valp);
> > #endif
>
> etc.
>
> I'd prefer to return the fetched value and have an out of band error
> check.
>
> With this API the routine does a userland load, then a store into kernel
> memory.  Then the kernel needs to reload that value.  On modern CPUs
> memory accesses tend to be sloow.

So the out-of-band error check becomes the slow memory write?
I don't see it as a problem as the data would presumably be written to
the write-back cached's stack (besides, if the function is short, LTO
will eliminate it).

FWIW, I suspect the most "efficient" way to return the value+error as
a struct - ABIs would return the pair in registers or, failing that,
pass a single stack address - but I don't think this interface should
go there.

Re: RFC: New userspace fetch/store API

[Andrew Cagney]

On Sun, 24 Feb 2019 at 10:38, Jason Thorpe  wrote:
>
>
> > On Feb 24, 2019, at 7:05 AM, Andrew Cagney  wrote:
> >
> > Can there be an inefficient default, implemented using something like
> > copy{in,out}() (yes I note the point about alignment).  I sometimes
> > wonder of NetBSD's lost its way in terms of portability - it seems to
> > be accumulating an increasing number of redundant yet required
> > primitives.
>
> Honestly, it’s not adding that many — most of the ones specified in my 
> proposal were required before (to varying degrees) just with different names 
> and return semantics.  And most of the “new” ones in my proposal as simple 
> aliases.  Furthermore, on the two implementations I’ve done so far, a great 
> deal of the code is macro-ized to reduce the redundant typing, and even the 
> “intrsafe” ones share most of the code by using a different (macro-ized) 
> prologue and then jumping into the body of the non-intrsafe variant.  It’s 
> really not that much code.

Right.. But can it be done once in generic code using information
available while compiling?

> As far as “inefficient default” … depending on how copyin / copyout are 
> implemented, it’s also an issue of correctness.  At least the 32-bit and 
> 64-bit variants would need to be atomic with respect to other loads / stores 
> to the same address.  This is one property that I need for my own use.

And that's a hard requirement.  What of the other way around - copy*()
implemented using fetch_*().



> If there really is a concern about the number of operations here,I would be 
> more than happy to drop the _8 and _16 (and thus char and short) widths from 
> the formal API… but then other MI code would need to change to lose that 
> functionality (in the case of the profiling code that uses fuswintr() and 
> suswintr()), or to use heavier means of doing the same thing (e.g. copyin or 
> copyout of a single byte).
>
> -- thorpej
>

Re: RFC: New userspace fetch/store API

[Andrew Cagney]

On Sat, 23 Feb 2019 at 18:26, Jason Thorpe  wrote:

> A project I’m working on has a need for a proper “int” size fetch / store on 
> all platforms, and it seems best to just tidy the whole mess up.  So, I am 
> proposing a new replacement user fetch/store API.

Can you be more specific on why "int" rather than a more abstract type
(we've things like size_t ptrdiff_t et.al. all kicking around) - I'm
curious.

> I have implemented the new API for alpha and amd64, and am putting together a 
> test harness to exercise all aspects of the new API.  Once that is done, I’ll 
> tackle the remaining architectures.

> The implementation is entirely in architecture-dependent code.  The following 
> fetch primitives must be supplied:

As a corollary to KRE's point, why?

Can there be an inefficient default, implemented using something like
copy{in,out}() (yes I note the point about alignment).  I sometimes
wonder of NetBSD's lost its way in terms of portability - it seems to
be accumulating an increasing number of redundant yet required
primitives.

> int ufetch_8(const uint8_t *uaddr, uint8_t *valp);

To follow through with Edgar Fuß's point.  If I were looking for a
function to transfer a uint8_t I'd expect it to be called
fetch_uint8().

> int ufetch_16(const uint16_t *uaddr, uint16_t *valp);
> int ufetch_32(const uint32_t *uaddr, uint32_t *valp);
> #ifdef _LP64
> int ufetch_64(const uint64_t *uaddr, uint64_t *valp);
> #endif
>
> The following aliases must also be provided, mapped to the appropriate 
> fixed-size primitives:
>
> int ufetch_char(const unsigned char *uaddr, unsigned char *valp);
> int ufetch_short(const unsigned short *uaddr, unsigned short *valp);
> int ufetch_int(const unsigned int *uaddr, unsigned int *valp);
> int ufetch_long(const unsigned long *uaddr, unsigned long *valp);
> int ufetch_ptr(const void **uaddr, void **valp);

I do find passing pointers as both the addr and the dest confusing;
but you've added a const which should help get the order right.

Re: awge0 and 100mb?

[Andrew Cagney]

On 26 April 2018 at 11:26, Manuel Bouyer <bou...@antioche.eu.org> wrote:
> On Thu, Apr 26, 2018 at 11:20:22AM -0400, Andrew Cagney wrote:
>> > I tested on a olimex lime2 (A20 with a realtek Gbe PHY), it doens't make a
>> > difference either.
>>
>> Unfortunately, it didn't seem to help my 100mb connection.
>> If there's something specific to look at let me know.
>
> What board and PHY do you have ?

ODROID-C1+ as in?

CPU : AMLogic S805
MEM : 1024MB (DDR3@792MHz)

amlogicio0 at mainbus0
awge0 at amlogicio0: Gigabit Ethernet Controller
awge0: interrupting on irq 40
awge0: Ethernet address:
rgephy0 at awge0 phy 0: RTL8169S/8110S/8211 1000BASE-T media interface, rev. 6
rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, auto
rgephy1 at awge0 phy 1: RTL8169S/8110S/8211 1000BASE-T media interface, rev. 6
rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, auto

Andrew

awge0 and 100mb?

[Andrew Cagney]

Hi,

Anyone had, er, fun trying to get awge0 to connect at 100mb?

I foolishly changed the hub that my working odroid c1 was connected to
and after that packets refused to flow (other end didn't see them).
Switching it back restored connectivity.  After some thrashing about I
noticed that while the good hub does 1gb the bad one does only 100mb
(confirmed by ifconfig).  It could be the hubs, but given that when
they are connected packets flow, I'd suspect not.

Currently working up a kernel more recent than NetBSD 8.0_BETA
(ODROID-C1) #1: Wed Mar  7 09:03:51 UTC 2018

Andrew

Re: gcc: optimizations, and stack traces

[Andrew Cagney]

Two things come to mind:

- was this the innermost (newest) frame?  If it wasn't something
earlier could be the problem
- is the dwarf debug info being used, or is it relying on heuristics
(annoyingly I can't spot an easy way to tell)

and when this happens, 'info frame' may help diagnose things.

Looking at:

>
> void
> kernfs_get_rrootdev(void)
> {
> static int tried = 0;
>
> if (tried) {
> /* Already did it once. */
> return;
> }
> tried = 1;
>
> if (rootdev == NODEV)
> return;
> rrootdev = devsw_blk2chr(rootdev);
> if (rrootdev != NODEV)
> return;
> rrootdev = NODEV;
> printf("kernfs_get_rrootdev: no raw root device\n");
> }

I get:

043c :
 43c:   8b 05 00 00 00 00   mov0x0(%rip),%eax# 442

 442:   85 c0   test   %eax,%eax
 444:   75 2e   jne474 
 446:   c7 05 00 00 00 00 01movl   $0x1,0x0(%rip)# 450

 44d:   00 00 00
 450:   48 8b 3d 00 00 00 00mov0x0(%rip),%rdi# 457

 457:   48 83 ff ff cmp$0x,%rdi
 45b:   74 17   je 474 
 45d:   55  push   %rbp
 45e:   48 89 e5mov%rsp,%rbp
 ...

and has CFI (readelf --debug-dump=frames-interp
amd64/sys/arch/amd64/compile/GENERIC/kernfs_vfsops.o):

01a4 0028 01a8 FDE cie=
pc=043c..0484
   LOC   CFA  rbp   ra
043c rsp+8u c-8
045e rsp+16   c-16  c-8
...

so, until the push, the CFI has't specified RBP, but a reasonable
interpretation s current value.  So this, to me looks ok.

Re: gcc: optimizations, and stack traces

[Andrew Cagney]

On 23 February 2018 at 03:41, Maxime Villard  wrote:

> Many of our ASM functions don't push frames, but that's a different issue.

/me mumbles something about the assembler needing to be marked up with
.cfi directives

Re: starting dhclient on odroid-c1 (arm)

[Andrew Cagney]

>> > I fixed this in -current, but probably didn't ask for pullup yet (the
>> > change breaking it only recently got pulled up in #456).
>>
>>
>> Ah, I'll wait for more branch changes then try again.  Thanks.
>
> It has been pulled up last night.

Yes, working now.  Thanks.

Re: starting dhclient on odroid-c1 (arm)

[Andrew Cagney]

On 12 January 2018 at 09:32, Martin Husemann <mar...@duskware.de> wrote:
> On Fri, Jan 12, 2018 at 09:27:47AM -0500, Andrew Cagney wrote:
>> # uname -a
>> NetBSD  8.0_BETA NetBSD 8.0_BETA (ODROID-C1) #1: Wed Jan 10 00:55:46
>> EST 2018  evbearmv7hf-el/sys/arch/evbarm/compile/ODROID-C1 evbarm
>> # /etc/rc.d/dhclient onestart
>> Starting dhclient.
>> # panic: kernel diagnostic assertion "if_is_mpsafe(ifp)" failed: file
>> "/home/samsung/netbsd-build/8/src/sys/dev/ic/dwc_gmac.c", line 841
>
> I fixed this in -current, but probably didn't ask for pullup yet (the
> change breaking it only recently got pulled up in #456).


Ah, I'll wait for more branch changes then try again.  Thanks.

starting dhclient on odroid-c1 (arm)

[Andrew Cagney]

# uname -a
NetBSD  8.0_BETA NetBSD 8.0_BETA (ODROID-C1) #1: Wed Jan 10 00:55:46
EST 2018  evbearmv7hf-el/sys/arch/evbarm/compile/ODROID-C1 evbarm
# /etc/rc.d/dhclient onestart
Starting dhclient.
# panic: kernel diagnostic assertion "if_is_mpsafe(ifp)" failed: file
"/home/samsung/netbsd-build/8/src/sys/dev/ic/dwc_gmac.c", line 841
cpu2: Begin traceback...
0xbe973934: netbsd:db_panic+0xc
0xbe97394c: netbsd:vpanic+0x1b0
0xbe973964: netbsd:__udivmoddi4
0xbe9739c4: netbsd:dwc_gmac_start+0x314
0xbe9739ec: netbsd:dwc_gmac_ioctl+0x5c
0xbe973b34: netbsd:in6_update_ifa1+0x7b8
0xbe973b5c: netbsd:in6_update_ifa+0x40
0xbe973cbc: netbsd:in6_ifattach+0x3cc
0xbe973cd4: netbsd:in6_if_up+0x18
0xbe973cf4: netbsd:if_up_locked+0x7c
0xbe973d1c: netbsd:ifioctl_common+0x160
0xbe973d3c: netbsd:ether_ioctl+0x194
0xbe973d64: netbsd:dwc_gmac_ioctl+0x38
0xbe973e6c: netbsd:doifioctl+0x6f0
0xbe973f34: netbsd:sys_ioctl+0x274
0xbe973fac: netbsd:syscall+0x104
cpu2: End traceback...

Re: ptrace(2) interface for hardware watchpoints (breakpoints)

[Andrew Cagney]

On 15 December 2016 at 13:23, Kamil Rytarowski  wrote:

> BTW. I'm having some DWARF related questions, if I may reach you in a
> private mail?
>

Better is http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org
so you don't have me as a bottle neck.

Andrew

Re: ptrace(2) interface for hardware watchpoints (breakpoints)

[Andrew Cagney]

On 15 December 2016 at 13:22, Eduardo Horvath <e...@netbsd.org> wrote:

>
> On Thu, 15 Dec 2016, Andrew Cagney wrote:
>
> > Might a better strategy be to first get the registers exposed, and then,
> if
> > there's still time start to look at an abstract interface?
>
> That's one way of looking at it.
>
> Another way is to consider that watchpoints can be implemented through
> careful use of the MMU.
>
>
Yes, HP implemented something like this with wildebeast (gdb fork) on HP-UX.


> > and now lets consider this simple example, try to watch c.a in:
> >
> > struct { char c; char a[3]; int32_t i; int64_t j; } c;
> >
> > Under the proposed model (it looks a lot like gdb's remote protocol's Z
> > packet) it's assumed this will allocate one watch-point:
> >
> > address=, size=3
>
> So when you register the watchpoint, the kernel adds the address and size
> to an internal list and changes the protection of that page.  If it's a
> write watchpoint, the page is made read-only.  If it's a read watchpoint,
> it's made invalid.
>
> The userland program runs happily until it tries to access something on
> the page with the watchpoint.  Then it takes a page fault.
>
>
Threads cause the theory to cough up a few fur balls, but nothing fatal.


> The fault handler checks the fault address against its watchpoint list,
> and if there's a match, send a ptrace event and you're done.
>
> If it doesn't match the address, the kernel can either map the address in
> and single step, set a breakpoint on the next instruction, or emulate the
> instruction, and then protect the page again to wait for the next fault.
>
> It has a bit more overhead than using debug registers, but it scales a lot
> better.
>

Agreed.  There are many things that can be implemented to improve user land
debugging.  Simple watch-points (especially if they work on the kernel) are
a good starting point.

Andrew

Re: ptrace(2) interface for hardware watchpoints (breakpoints)

[Andrew Cagney]

On 13 December 2016 at 12:16, Kamil Rytarowski  wrote:

> >> 5. Do not allow to mix PT_STEP and hardware watchpoint, in case of
> >> single-stepping the code, disable (it means: don't set) hardware
> >> watchpoints for threads. Some platforms might implement single-step with
> >> hardware watchpoints and managing both at the same time is generating
> >> extra pointless complexity.
>
>
Is this wise?  I suspect it might be better to just expose all the hairy
details and let the client decide if the restriction should apply.
(to turn this round, if the details are not exposed, then clients will
wonder why their platform is being crippled).


> > I don't think I see how "extra pointless complexity" follows.
> >
>
> 1. At least in MD x86 specific code, watchpoint traps triggered with
> stepped code are reported differently to those reported with plain steps
> and also differently to plain hardware watchpoint traps. They are 3rd
> type of a trap.
>
> 2. Single stepping can be implemented with hardware assisted watchpoints
> (technically breakpoints) on the kernel side in MD. And if so, trying to
> apply watchpoints and singlestep will conflict and this will need
> additional handling on the kernel side.
>
> To oppose extra complexity I propose to make stepping and watchpoints
> separable, one or the other, but not both.

Re: ptrace(2) interface for hardware watchpoints (breakpoints)

[Andrew Cagney]

[see end]

On 13 December 2016 at 12:16, Kamil Rytarowski  wrote:

> On 13.12.2016 04:12, Valery Ushakov wrote:
> > On Tue, Dec 13, 2016 at 02:04:36 +0100, Kamil Rytarowski wrote:
> >
> >> The design is as follows:
> >>
> >> 1. Accessors through:
> >>  - PT_WRITE_WATCHPOINT - write new watchpoint's state (set, unset, ...),
> >>  - PT_READ_WATCHPOINT - read watchpoints's state,
> >>  - PT_COUNT_WATCHPOINT - receive the number of available watchpoints.
> >
> > Gdb supports hardware assisted watchpoints.  That implies that other
> > OSes have existing designs for them.  Have you studied those existing
> > designs?  Why do you think they are not suitable to be copied?
> >
>
> They are based on the concept of exporting debug registers to tracee's
> context (machine context/userdata/etc). FreeBSD exposes MD-specific
> DBREGS to be set/get by a user, similar with Linux and with MacOSX.
>
> GDB supports hardware and software assisted watchpoints. Software ones
> are stepping the code and checking each instruction, hardware ones make
> use of the registers.
>
> I propose to export an interface that is not limited to one type of
> hardware assisted action, while it can be fully used for hardware
> watchpoints (if CPU supports it). This interface will abstract
> underlying hardware specific capabilities with a MI ptrace(2) calls (but
> MD-specific ptrace_watchpoint structure).
>
> These interfaces are already platform specific and aren't shared between
> OSes.
>
>
That isn't true (or at least it shouldn't).

While access to the registers is OS specific, the contents of the
registers, and their behaviour is not.  Instead, that is specified by the
Instruction Set Architecture.
For instance, FreeBSD's gdb/i386fbsd-nat.c uses the generic
gdb/x86-nat.c:x86_use_watchpoints() code.



> Some time ago I checked and IIRC the only two users of these interfaces
> were GDB and LLDB, I implied from this that there is no danger from
> heavy patching 3rd party software.


I'm not sure how to interpret this.  Is the suggestion that, because there
are only two consumers, hacking them both will be easy; or something else?
I hope it isn't.  Taking on such maintenance has a horrendous cost.

Anyway, lets look at the problem space.  It might help to understand why
kernel developers tend to throw up their hands.

First lets set the scene:

- if we're lucky we have one hardware watch-point, if we're really lucky
there's more than one
- if we're lucky it does something, if we're really lucky it does what the
documentation says

which reminds me:

- if we're lucky we've got documentation, if we're really lucky we've
correct and up-to-date errata explaining all the hair brained interactions
these features have with other hardware events

and now lets consider this simple example, try to watch c.a in:

struct { char c; char a[3]; int32_t i; int64_t j; } c;

Under the proposed model (it looks a lot like gdb's remote protocol's Z
packet) it's assumed this will allocate one watch-point:

address=, size=3

but wait, the hardware watch-point registers have a few, er, standard
features:

- I'll be kind, there are two registers
- size must be power-of-two (lucky size==4 isn't fixed)
- address must be size aligned (lucky addr & 3 == 0 isn't fixed)
- there are separate read/write bits (lucky r+w isn't fixed)

so what to do?  With this hardware we can:

- use two watch-point registers (making your count meaningless), so that
accesses only apply to the address in question

- use one watch-point register and over-allocate the address/size and then
try to figure out what happened
For writes, a memcmp can help, for reads, well you might be lucky and have
a further register with the access address, or unlucky and find yourself
disassembling instructions to figure out what the address/size really was

Now, lets consider what happens when the user tries to add:

   , size=8

depending on where all the balls are (above decision, and the hardware),
that may or may not succeed:

  - 32-bit hardware probably limits size<=4, so above would require two
registers
  - even if not, ,size=3 may have already used up the two registers

Eww.

Might a better strategy be to first get the registers exposed, and then, if
there's still time start to look at an abstract interface?

Andrew

Re: nick-nhusb merge coming soon

[Andrew Cagney]

On 28 April 2016 at 06:56, Paul Goyette  wrote:
> On Wed, 13 Apr 2016, Paul Goyette wrote:
>
>> Does this include xhci support for USB3 (ie, removal of the "experimental"
>> tag)?
>
>
> FWIW, I finally got around to checking the status of USB3 on my machine.
>
> Firstly, let me note that I do not have any USB3 peripherals.  My only USB3
> equipment is the USB3 support on the motherboard itself.
>
> With an older 7.99.26 kernel, USB is totally screwed if I enable the USB3
> ports.  In order for _any_ USB device (ie, my mouse and keyboard) to work, I
> need to disable USB3 support in the BIOS.
>
> I'm happy to note that this restriction no longer exists!  With a kernel
> built from today's sources, the system boots just fine with all USB3 BIOS
> settings enabled, and the USB keyboard and mouse function normally.

If you have an install image USB stick handy, does it work when
plugged into a USB3 port?
I keep being tripped up by that one, and for multiple OSs.  Guess I
should give it another go :-)

Re: Patch: CPU RNG framework with stub x86 implementation

[Andrew Cagney]

On 10 January 2016 at 21:08, Thor Lancelot Simon  wrote:
> As requested - here's just "cpu_rng" itself for review.  I believe this
> version addresses all comments received so far except that I _like_ the
> construct "size_t cnt = ; type_t foo[x]" so I've retained
> that.

According to that ever reliable wikipedia :-)

+ size_t cnt = 2 * RND_ENTROPY_THRESHOLD / sizeof(cpu_rng_t);
+ cpu_rng_t buf[cnt];

variable-length arrays were added in C99, but subsequently, in C11,
were relegated to a conditional feature which implementations are not
required to support.

Re: Guidelines for choosing MACHINE MACHINE_ARCH?

[Andrew Cagney]

On 24 June 2015 at 23:38, David Holland dholland-t...@netbsd.org wrote:
 On Wed, Jun 24, 2015 at 04:01:24PM -0700, Matt Thomas wrote:

I agree that evb* is confusing and increasingly meaningless and
would like to see us transition away from it.
  
   I contend that moving to sys/arch/cpu is incorrect which there
   are multiple MACHINE values for that CPU.  sys/tem/mips (haha!) or
   sys/platform/mips (yuk) or sys/arch/cpusys or something better.

It's all too confusing.  I'd like to be able to do:

   ./build.sh rpi

but instead I use:

  ./build.sh -... evbarm

and likely build far more than I need.

 [...]

 That said, given the ability to have multiple userland builds for a
 given port, which is more or less necessary now with all the arm
 variants, there's no real reason to have more than one port per cpu
 type. One of the traditional criteria was whether you needed a
 different kernel; but nowadays having different configs (as evbarm
 does) is good enough. Similarly, even if you need a different
 bootloader we should be able to sort that out without having a whole
 separate port.

This probably illustrates why 'cpu' has become so overloaded.

When building userland,  cpu identifies:

- a system-call convention for the architecture
- user instruction set architecture (to borrow an IBM term) + calling
convention (application binary interface)
- and?

How do I build a 32-bit LE soft-float Arm9 userland?  Or more more
seriously, how can I build just a arm7hf targeted compiler and user
land?

For the kernel cpu means a lot more:

- first, the kernel can adopt its own ISA/ABI (64-bit kernel, 32-bit userland)
- chip-family specific code for dealing with VM and context switches
- other stuff I can't think of

(IBM called this the operating environment architecture).

Perhaps, rather than trying to re-organize the source tree,  the first
step is to look at build.sh.  For instance:

- clarify/simplify the terminology (MACHINE_ARCH / -e seems to be the ISA/ABI)
perhaps ./build.sh -e arm7hf tools sets builds my arm7hf userland,
it isn't clear

- allow finer grained machines or platforms so I can have
./build.sh -m rpi do the right thing

Only once the model seems to be working, start looking at the source code.

Andrew

 So in the long run I'd say it should just go by cpu type. But as
 mashing together e.g. everything in sys/arch/arm and everything in
 sys/arch/evbarm would add a lot of entropy, I think we need a stronger
 model for the substructure of these directories than we currently
 have.

 (Also, I think MD stuff should be distributed around the tree more; we
 should e.g. keep MD virtual memory stuff in sys/uvm/arch. And we
 should keep MD system call stuff in sys/syscall/arch or something like
 that, although this first requires splitting sys/syscall out of
 sys/kern. Splitting things up this way requires some attention to
 MD-level interfaces between subsystems; but having gone through that
 with a research kernel I'd say it's beneficial.)

 all this is moot when we can't cvs rename though...

 --
 David A. Holland
 dholl...@netbsd.org

Re: lua kernel library?

[Andrew Cagney]

On 18 June 2015 at 05:58, Valery Ushakov u...@stderr.spb.ru wrote:
 On Wed, Jun 17, 2015 at 09:55:31 -0400, Andrew Cagney wrote:

 pulling stuff like memory into Lua has proven relatively painless
 (which reminds me, how do I silently detect that db_read_bytes
 failed).

 Do you mean detecting - detecting the fault, and/or silently - not
 printing Faulted in DDB...?

Both :-)

- get some sort of status indication that a memory read was invalid

- not have the error message printed

my code looks like:

uint8_t byte;
//printf(%zx/%zu, addr, addr);
db_read_bytes(addr, sizeof(byte), byte);
//printf( %02x/%u\n, byte, byte);
lua_pushinteger(L, byte);
return 1;

if the address is invalid, something inside of db_read_bytes prints a
message but still returns.  Not exactly correct :-)

 ISTR, we always print Faulted in DDB.  To detect the fault you
 probably need to use setjmp/db_recover around the access.

Ah, I'll dig further :-)

 -uwe

Re: lua kernel library?

[Andrew Cagney]

On 17 June 2015 at 02:43, matthew green m...@eterna.com.au wrote:

 what sort of support from ddb do you need?  ddb supports run-time
 addition of commands, so i imagine that with the right glue you
 could easily add a lua ddb command with a module.

That's useful to know.   Since I'm playing, I hacked ddb directly.

So far the only notable tweak I've made is to add a prompt parameter
to db_readline.  Otherwise I found myself looking at:

db lua
db

and wondering if my code worked :-)   However, before worrying about
that, my lua main-loop should be inverted (so it is closer to lua.c) -
my proof-of-concept followed what's found in the book.

Going forward, the most interesting thing will likely be hacking
(bypassing) ddb so that events such as breakpoints and watchpoints are
directly pushed (injected) into a Lua instance.  So far, the reverse,
pulling stuff like memory into Lua has proven relatively painless
(which reminds me, how do I silently detect that db_read_bytes
failed).

Oh, it might be nice if readline behaved more like readline :-)  It
doesn't seem to under QEMU.

Andrew

Re: lua kernel library?

[Andrew Cagney]

On 17 June 2015 at 00:48, Lourival Vieira Neto lourival.n...@gmail.com wrote:
 Is the kernel's version of Lua available as a library?

 You can use it as a regular kernel module. Take a look at this patch
 [1] which adds Lua to NPF, as an example.

 thanks, I'd looked briefly at the lua et.al. modules.

 Please notice that the modules already present in base work like Lua
 libraries (that is, they are called by Lua) and npf_lua works like a
 host program (calling Lua). I think the last is more appropriate to
 your use case.

I'm not so sure.   If I create a module and then have that bind to
ddb and lua then yes.
However, if I try to make this work before modules have even been
loaded then, no.

 (...)
 to sys/ddb/files.ddb is not the best way to link Lua into DDB

 My understanding of this general approach is that it would make DDB
 dependent on a [possibly loadable] kernel module?  Or perhaps I can
 add a lua pseudo device and start calling the underlying library
 directly?

 To me an in-kernel debugger needs to be both largely standalone and
 callable from every very early in the boot process.  For instance,
 just after the serial console's initialized and showing signs of
 working.

 Then, I think you should just compile lua(4) statically instead of
 reimplementing it tied to DDB.

I'm not sure I'm following.  Perhaps we're agreeing?

My lua binding needs access to DDB's code, and the DDB code needs
access to my Lua instance.  This is why I added the lua files to
sys/ddb/files.ddb.  It seemed tedious.   I suspect it would be easier
to create lua.a (built for the kernel) and have both the lua module
and (optionally) ddb+kernel link against it.

 (This is also why I need to change my hack so that it uses a static
 buffer for Lua's heap; using the kernel to allocate memory when
 debugging the kernel's memory allocator doesn't tend to work very well
 :-)

 Then, you should just call klua_newstate(9) passing your custom
 allocator =). BTW, perhaps Luadata [2] can help you to hack memory.

 [2] https://github.com/lneto/luadata/

Yes.  For the moment I've a really simple binding:

lua m=setmetatable({},{ __index=function(table,addr) return
db_peek(addr); end})
lua print(m[db_debug_buf+1],m[db_debug_buf+2],m[db_debug_buf+3])
697670

 What's your plan for DDB+Lua? =)

 literally, to see what happens when you put lua and dwarf in a kernel
 (see BSDCan)

 Pretty cool! Are the slides publicly available in somewhere? (I couldn't 
 find.)

They should appear shortly.  I've also put them here
https://bitbucket.org/cagney/netbsd/downloads

 Moreover, I notice that you've found a bug on the usage of snprintf.
 Was this the only one? Please report it next time! =)

That was the only problem I found, which is impressive.  BTW, is lua's
testsuite run against the in-kernel lua module?

Andrew

lua kernel library?

[Andrew Cagney]

Hi,

Is the kernel's version of Lua available as a library?  I suspect adding this:

fileddb/db_lua.cddb
makeoptions ddb CPPFLAGS+=-I$S/../external/mit/lua/dist/src
makeoptions ddb CPPFLAGS+=-I$S/sys
makeoptions ddb CPPFLAGS+=-Wno-error=cast-qual
makeoptions ddb CPPFLAGS+=-Wno-error=shadow
file../external/mit/lua/dist/src/lapi.c ddb
file../external/mit/lua/dist/src/lcode.c ddb
...

to sys/ddb/files.ddb is not the best way to link Lua into DDB

Andrew

Re: lua kernel library?

[Andrew Cagney]

On 16 June 2015 at 13:52, Lourival Vieira Neto lourival.n...@gmail.com wrote:
 Hi Andrew,

 Is the kernel's version of Lua available as a library?

 You can use it as a regular kernel module. Take a look at this patch
 [1] which adds Lua to NPF, as an example.

thanks, I'd looked briefly at the lua et.al. modules.

 [1] http://www.netbsd.org/~lneto/pending/0005-added-npf_ext_lua.patch

 (...)
 to sys/ddb/files.ddb is not the best way to link Lua into DDB


My understanding of this general approach is that it would make DDB
dependent on a [possibly loadable] kernel module?  Or perhaps I can
add a lua pseudo device and start calling the underlying library
directly?

To me an in-kernel debugger needs to be both largely standalone and
callable from every very early in the boot process.  For instance,
just after the serial console's initialized and showing signs of
working.

(This is also why I need to change my hack so that it uses a static
buffer for Lua's heap; using the kernel to allocate memory when
debugging the kernel's memory allocator doesn't tend to work very well
:-)

 What's your plan for DDB+Lua? =)

literally, to see what happens when you put lua and dwarf in a kernel
(see BSDCan)

 Regards,
 --
 Lourival Vieira Neto

Re: Removing ARCNET stuffs

[Andrew Cagney]

On 8 June 2015 at 13:18, Anders Magnusson ra...@ludd.ltu.se wrote:
 David Holland skrev den 2015-06-08 19:06:

 On Mon, Jun 08, 2015 at 04:15:15PM +0200, Anders Magnusson wrote:
printfing from the back of the front end is definitely totally wrong
in other ways that need to be rectified first :(
Hm, I may be missing something, but what is wrong?
Where should you print it out otherwise?

 I would say the debug information ought to be attached to the
 representation nodes it describes as it moves through the backend;
 otherwise it's easily upset by fairly simple transformations.

 It's simpler than that :-)

 Debug info for all data declarations etc are printed out when found,
 so are the declarations themselves.  No reason to keep them.

Traditional stabs, with their very limited capacity to describe whats
going on don't exactly help (there are extensions), but they aren't
really the problem.

As you note, data values are printed by stabs.c:stabs_newsym() at the
start of a block.  For instance:

case AUTO:
cprint(0, \t.stabs \%s:%s\,%d,0, CONFMT ,%d\n,
sname, ostr, N_LSYM, (CONSZ)suesize, BIT2BYTE(s-soffset));
break;
case REGISTER:
cprint(0, \t.stabs \%s:r%s\,%d,0,%d,%d\n,
sname, ostr, N_RSYM, 1, s-soffset);
break;

it might be mostly correct at .O0 when on a statement boundary, but
not when there's an optimizer.  For instance:

- stack values never get written back to the stack
- multi-register values are never in contiguous registers

(which is why debuggers invariably print wrong values).  The compiler
needs to describe where values are, for failing that, describe that
the value was lost.

 Debug info for code follows the code itself. This is necessary, since
 the debug info must follow the reference to a place in the code stream,
 otherwise the debug info might refer to a label that is optimized away
 which is not acceptable by the assembler :-)

In stabs.c:stabs_line, there's what I'm going to call a gem:

void
stabs_line(int line)
{
if (inftn == 0)
return; /* ignore */

Part of debugger folk law is to set a breakpoint on a function the
debugger needs to do something like:

- find foo
- find foo's first stabn
- set a breakpoint on the stabn, and then cross fingers and hope that
the compiler hasn't sneezed

the above code, which suppresses line-number information in the
epilogue, explains why.

Even without optimization shooting us in the foot, a program that
stops (crashes) inside the epilogue (or initialization of a block, or
mid way through a statement) has wrong information(1) - the stack
isn't fully formed for instance (I suspect there was also folk law
saying debuggers should surreptitiously single-step to the next stabn
when they find the program stopped between two lines).

Andrew

(1) it may also have trouble unwinding, but that is another story

Re: Removing ARCNET stuffs

[Andrew Cagney]

Oops, s/epilogue/prologue/ in below

On 8 June 2015 at 15:15, David Holland dholland-t...@netbsd.org wrote:
 On Mon, Jun 08, 2015 at 07:18:24PM +0200, Anders Magnusson wrote:
   David Holland skrev den 2015-06-08 19:06:
   On Mon, Jun 08, 2015 at 04:15:15PM +0200, Anders Magnusson wrote:
  printfing from the back of the front end is definitely totally wrong
  in other ways that need to be rectified first :(
  Hm, I may be missing something, but what is wrong?
  Where should you print it out otherwise?
   
   I would say the debug information ought to be attached to the
   representation nodes it describes as it moves through the backend;
   otherwise it's easily upset by fairly simple transformations.
  
   It's simpler than that :-)
  
   Debug info for all data declarations etc are printed out when found,
   so are the declarations themselves.  No reason to keep them.

 ...except perhaps for, say, removing unused static variables :-)

   Debug info for code follows the code itself. This is necessary, since
   the debug info must follow the reference to a place in the code stream,
   otherwise the debug info might refer to a label that is optimized away
   which is not acceptable by the assembler :-)

 Ok, never mind then; was responding to what was posted rather than
 what actually exists...

 --
 David A. Holland
 dholl...@netbsd.org

Re: Removing ARCNET stuffs

[Andrew Cagney]

I'm clearly out-of-date regarding SSA, its nice to be corrected.

On 8 June 2015 at 09:06, Anders Magnusson ra...@ludd.ltu.se wrote:
 Andrew Cagney skrev den 2015-06-01 20:41:

 I do not understand why either of those choices need to be taken.
 Pcc has a reasonable intermediate representation, which in the optimizer
 is converted to SSA form, hammered on, and converted back.  This is
 done while retaining the intermediate representation, which is no problem.

 I'm being fast and loose.  My reading of the code was that debug info
 was being generated by the back of the front end (very roughly
 gimplify in this diagram of GCC
 https://gcc.gnu.org/projects/tree-ssa/#ssa).   It was pretty much hard
 wired printfs, and explained to me why -g -O wasn't supported.

 printf's are only used for data (which is spit out directly), not code.
 All code is dealt with function-wise, otherwise things like the register
 allocator would not work.

 Unless, I guess, what you're talking about is throwing away the
 existing backend entirely and writing a new SSA-based one, in which
 case I'd gently suggest that this is a large project :-)

 Exactly :-(

 What is wrong with the existing backend?

As you state, the representation gets taken into and then out of SSA.
Why bother; at least for the converting to-ssa side?

Andrew

Re: retrocomputing NetBSD style

[Andrew Cagney]

 GAW Wrote:
  I really don't understand anyone who has the desire to try to run
  build.sh on a VAX-750 to build even just a kernel, let alone the
  whole distribution.

 I recall a time where NetBSD/vax was broken for a long time because
 everyone was cross-building; as soon as a native build was attemped,
 the brokenness showed up.

 I native build on _everything_.  If it can't native build, it isn't
 really part of my stable, so to speak.

 Yes, there is that issue!

 See, for instance, my recent posts comparing assembler output from
 kernel compiles done by the same compiler when run on amd64 vs. i386.

 However those are the kinds of bugs one might hope can be caught by
 decent enough regression tests of the compiler and its toolchain.

Could always set up the machine (or simulator) as a remote target and
run the cross-compiler against that; or perhaps that is a lost art :-(

Andrew

Re: Removing ARCNET stuffs

[Andrew Cagney]

On 3 June 2015 at 01:56, David Holland dholland-t...@netbsd.org wrote:
 On Mon, Jun 01, 2015 at 02:41:22PM -0400, Andrew Cagney wrote:
 To my mind, and I'm assuming a pure SSA compiler design, having SSA
 forces issues like: [...]
   
I'm missing something; SSA is just a style of program representation.
  
   Yes.  Lets think of Static Single Assignment as the pure academic theory.

 Ok... although calling it a theory is a bit much?

Perhaps we can decide that by counting the number research papers and
PHDs its generated :-)

 It's just a scheme
 for laying out program representations. A useful one for various
 reasons; but it's not itself a representation and it doesn't have much
 effect on the rest of the architecture of the compiler backend, except
 maybe for naive backends.

Right.  Is that bad?

Keep in mind that I think of RTL, based on having to on rare occasions
prod it, as little more than glorified assembler.  And even a little
GCC probably corrupts my point-of-view :-)

One of the debates for GCC was over doing as much as possible using
the SSA representation vs keeping lots of [pre-existing] smarts in RTL
 Given GCC's architecture and the amount of existing intellectual and
emotional investment in RTL,  it was quickly off the table.

 Or so it seems to me anyway, but I've been up to my ears in compiler
 hokum in connection with a $WORK project for several years now.

   PCC, to the best of my knowledge is still in the [very early] planning
   stages.  One of its design choices would be to go pure SSA.  Another
   option, closer to GCC (RTL), would be to retain existing code-gen
   passes.  Tough choices.

 I'm not sure why it's such a big deal, except that everything in gcc
 is a big deal because gcc is such a mess inside.

From my reading, the goal was to keep *all* existing architectures
working.  Consequently, SSA was wedged into GCC in front of RTL.
Thus preserving all the legacy.

From what your saying, PCC won't suffer from this and can integrate it
directly into the existing framework.  Much easier.

 Moving an existing representation to SSA just requires adding phi
 nodes to the representation, writing a check pass to enforce the
 single assignment property, and updating existing passes to maintain
 the property -- in a language without proper algebraic data types this
 is a bigger deal than otherwise but it does not seem like a
 particularly major undertaking. Unless the representation is totally
 wrong in other ways that need to be rectified first.

 Maybe I'm missing something, or maybe I'm coming to this from a
 strange point of view.

Are you talking about a hypothetical future where pcc's backend grows
an (additional, or replacement) SSA-based layer? Or is pcc's
optimizing backend already SSA-based (per above, I don't know one way
or the other) and you're talking about making that preserve debug
info?
   
Either way, it seems like adding DWARF support to the non-optimizing
backend is an orthogonal issue.
  
   I'm being fast and loose.  My reading of the code was that debug info
   was being generated by the back of the front end (very roughly
   gimplify in this diagram of GCC
   https://gcc.gnu.org/projects/tree-ssa/#ssa).   It was pretty much hard
   wired printfs, and explained to me why -g -O wasn't supported.

 printfing from the back of the front end is definitely totally wrong
 in other ways that need to be rectified first :(

Or view it as a clean slate.

When SSA was added to GCC there was a perception that debugging became
worse.  For instance, RTL always assigned things to contiguous
registers so producers and consumers were both designed to assume
this.  SSA removed that limitation; it took longer for things to get
back in sync.  Part of the fallout was -fvar-tracking, and one of
the great GCC debates of 2007.  GCC, with RTL, of course had more
than double the challenge.

While PCC should have nothing approaching those problems, you can
perhaps see why I wonder if focusing first on SSA would be better.
Fortunately it isn't my call.

Andrew

Re: Removing ARCNET stuffs

[Andrew Cagney]

On 1 June 2015 at 02:15, Anders Magnusson ra...@ludd.ltu.se wrote:
 Andrew Cagney skrev den 2015-06-01 03:24:

 systems and generates reasonable code.  Unfortunately, and sorry PCC
 (stabs, really?),

 Feel free to add dwarf, the source is out there, and it wouldn't be
 especially difficult to do it.  I just haven't had time.
 Stabs was for free :-)

I'm not so sure (a year back I looked at the code with that in mind),
and wonder if any quick hack would end up being opportunity lost.

PCC, as a classic C compiler, only generates debug information at
-O0.  This this is because the stabs code is restricted to the
un-optimized code generator path.  Having the backend restricted to
DWARF when '-O0' might just be ok, were it not for SSA (static single
assignment).

To my mind, and I'm assuming a pure SSA compiler design, having SSA
forces issues like:

- new back-ends that talk SSA (GCC still seems to be unable to
eliminate RTL); and one day generate debug information

- intermediate debug information paired with the SSA so that, at -O0,
front end - SSA - backend can generate debug information

- the expectation that -O -g works - untouched SSA results in code
far far worse than PCC at -O0 (the basic backend has a lot of smarts
that deserve credit)

Just the SSA side of things (if debugging is ignored) is a lot of work
(LLVM's solution was to largely ignore debugging.  I once asked
Lattner directly about this and he answered that he considered it a
back-end problem).

If PCC starts generating DWARF then be sure I'll find and report bugs;
however I think that would be a distraction.

Andrew

(I've got to admit that I wonder if C is the best language for an
optimizing compiler; but then, it is what we have)

Re: Removing ARCNET stuffs

[Andrew Cagney]

On 1 June 2015 at 12:54, David Holland dholland-t...@netbsd.org wrote:
 On Mon, Jun 01, 2015 at 11:41:38AM -0400, Andrew Cagney wrote:
systems and generates reasonable code.  Unfortunately, and sorry PCC
(stabs, really?),
   
Feel free to add dwarf, the source is out there, and it wouldn't be
especially difficult to do it.  I just haven't had time.
Stabs was for free :-)
  
   I'm not so sure (a year back I looked at the code with that in mind),
   and wonder if any quick hack would end up being opportunity lost.

 I have not looked at it, nor have I looked at pcc at all in a long
 time, so what I'm missing may just be otherwise obvious context, but:

   PCC, as a classic C compiler, only generates debug information at
   -O0.  This this is because the stabs code is restricted to the
   un-optimized code generator path.  Having the backend restricted to
   DWARF when '-O0' might just be ok, were it not for SSA (static single
   assignment).
  
   To my mind, and I'm assuming a pure SSA compiler design, having SSA
   forces issues like: [...]

 I'm missing something; SSA is just a style of program representation.

Yes.  Lets think of Static Single Assignment as the pure academic theory.

LLVM[Lattner et.al.] and GIMPLE[Novillo et.al.] are real world
implementations of that theory.
https://gcc.gnu.org/projects/tree-ssa/#ssa has a good diagram and is a
relevant read.

PCC, to the best of my knowledge is still in the [very early] planning
stages.  One of its design choices would be to go pure SSA.  Another
option, closer to GCC (RTL), would be to retain existing code-gen
passes.  Tough choices.

 Are you talking about a hypothetical future where pcc's backend grows
 an (additional, or replacement) SSA-based layer? Or is pcc's
 optimizing backend already SSA-based (per above, I don't know one way
 or the other) and you're talking about making that preserve debug
 info?

 Either way, it seems like adding DWARF support to the non-optimizing
 backend is an orthogonal issue.

I'm being fast and loose.  My reading of the code was that debug info
was being generated by the back of the front end (very roughly
gimplify in this diagram of GCC
https://gcc.gnu.org/projects/tree-ssa/#ssa).   It was pretty much hard
wired printfs, and explained to me why -g -O wasn't supported.

 Unless, I guess, what you're talking about is throwing away the
 existing backend entirely and writing a new SSA-based one, in which
 case I'd gently suggest that this is a large project :-)

Exactly :-(

Andrew


   (I've got to admit that I wonder if C is the best language for an
   optimizing compiler; but then, it is what we have)

 It is not, but one can do a lot worse. Plus if you try you end up in
 bootstrap hell like e.g. lang/ghc.

 --
 David A. Holland
 dholl...@netbsd.org

Re: Removing ARCNET stuffs

[Andrew Cagney]

   (oh and please delete C++ groff,  just replace it with that AWK script)

 which awk script? :-)

 (quite seriously, I've been looking for a while for an alternative to
 groff for typesetting the miscellaneous articles in base.

I was thinking of http://doc.cat-v.org/henry_spencer/awf/ which I
understand was adopted by MINIX.
I'm sure there are other choices.

Andrew

Re: Removing ARCNET stuffs

[Andrew Cagney]

On 1 June 2015 at 13:50, David Holland dholland-t...@netbsd.org wrote:
 but ignoring that -- who (other than apparently the gcc development
 team) is focusing on burning ram?

GNU, this is from the GNU coding standard; to me it explains some of
the design choices I find in many GNU utilities:

For example, Unix utilities were generally optimized to minimize
memory use; if you go for speed instead, your program will be very
different. You could keep the entire input file in memory and scan it
there instead of using stdio. Use a smarter algorithm discovered more
recently than the Unix program. Eliminate use of temporary files. Do
it in one pass instead of two (we did this in the assembler). 

Re: Removing ARCNET stuffs

[Andrew Cagney]

On 1 June 2015 at 15:13, Iain Hibbert plu...@ogmig.net wrote:
 On Mon, 1 Jun 2015, Andrew Cagney wrote:

 PCC, as a classic C compiler, only generates debug information at
 -O0.  This this is because the stabs code is restricted to the
 un-optimized code generator path.

 this is not actually the case btw, and I don't recall it being like that
 in the last few years either..

Like I mentioned in another reply, I'm being a little fast and loose.

The file cc/ccom/scan.l from
http://pcc.ludd.ltu.se/fisheye/browse/pcc/pcc/cc/ccom/scan.l?r=1.127
which I'm assuming is the C parser is doing this:

#define STABS_LINE(x) if (gflag  cftnsp) stabs_line(x)
...
\n{ ++lineno; STABS_LINE(lineno); }

which, I believe, is executing this:

cprint(1, \t.stabn %d,0,%d, STABLBL \n STABLBL :\n,
N_SLINE, line, stablbl, stablbl);

and this will will insert the stab into the assembler stream
(send_passt).  However, and here's the key thing, as best I can tell,
the stab isn't tied to an instruction, just a position in the
instruction stream.

If an optimizer so much as sneezes, re-ordering instructions for
instance, the information is wrong.

When the back-end goes to generate assembler, all it has, is a string.

In the case of variables.  They are generated at the start of a block,
so have no connection at all to the code.

perhaps I'm wrong?
Andrew

 regards,
 iain

Re: Removing ARCNET stuffs

[Andrew Cagney]

Yes, I'm being hypocritical :-)

On 31 May 2015 at 02:05, matthew green m...@eterna.com.au wrote:

 hi Andrew! :)

 Who is appalled to discover that pc532 support has been removed!

 get your GCC and binutils and GDB pals to put the support back
 in the toolchain and we'll have something to talk about :-)

 note that we've revived the playstation2 port now that its has
 had its toolchain components finally appear in mainline ... ;)

Right.  It takes time, effort and motivation to maintain code.  Not
people lamenting lost history :-)

--

For reference, this is what I wrote in 2004, when starting the process
of removing ns32k support when starting the process of removing NS32k
support.  It was two releases and roughly another year before the code
was completely removed.  It probably serves as a useful study given
what is being discussed here - extra infrastructure to prop up dieing
code just delayed the inevitable :-)

* END-OF-LIFE frame compatibility module

GDB's internal frame infrastructure has been completely rewritten.
The new infrastructure making it possible to support key new features
including DWARF 2 Call Frame Information.  To aid in the task of
migrating old configurations to this new infrastructure, a
compatibility module, that allowed old configurations to continue to
work, was also included.

GDB 6.2 will be the last release to include this frame compatibility
module.  This change directly impacts the following configurations:

h8300-*-*
mcore-*-*
mn10300-*-*
ns32k-*-*
sh64-*-*
v850-*-*
xstormy16-*-*

Re: Removing ARCNET stuffs

[Andrew Cagney]

On 30 May 2015 at 19:09, David Holland dholland-t...@netbsd.org wrote:
 The reason I floated the idea of forking is that an OS that's
 specifically intended to be a high-quality Unix for older hardware can
 make a different set of decisions (most notably, it can let C++ go
 hang) and this allows avoiding a wide and growing range of problems
 that currently affect NetBSD on old hardware. Meanwhile, it would also
 (one hopes) allow retaining a critical mass of retrocomputing
 enthusiasts in one place, as opposed to having them all gradually
 drift away in different directions.

Perhaps there are several, for want of a better phrase, niche plays here:

- remove C++ from base; Since when was UNIX's system compiler C++

Instead there should be small fast modern C compiler that runs on all
systems and generates reasonable code.  Unfortunately, and sorry PCC
(stabs, really?), this one currently doesn't exist, and no one seems
to have the time to make it exist, sigh.

This isn't to say that NetBSD shouldn't be cross-buildable using GNU
or LLVM, just that the above be the default compiler.

FreeBSD has clearly nailed itself to LLVM+LLDB and that means a C++
system compiler, and that, in turn means slow compiles on recent
machines with lots of ram.
And on a related note, the GNU tools - GCC, GOLD, GDB
(https://sourceware.org/gdb/wiki/cxx-conversion)  - which tend to be
used here - are are now all also members of the C++ Borg (is it only
time before binutils gets assimilated? :-) so if NetBSD does nothing,
it will find itself being lead down FreeBSD's C++ path.

(oh and please delete C++ groff,  just replace it with that AWK script)

- focus more on, and sorry for the marketing term, scalability

That is being able to run things on smaller and larger systems.  This
means more focus to algorithms and choices, and less focus on burning
ram like no tomorrow.

- look at, cough, simulators, cough, as a way to test and develop for
less mainstream architectures

The build process is sick, figuring out how to get it running on QEMU,
say, isn't.  Can this work, consistently, across all platforms:

qemu-system-XXX -nographic -kernel
sys/arch/XXX/compile/QEMU/netbsd -cdrom releasedir/... -sd0 disk.img

Andrew

Re: Removing ARCNET stuffs

[Andrew Cagney]

There's a complex tradeoff here

On 29 May 2015 at 12:09, J. Lewis Muir jlm...@imca-cat.org wrote:
 In Evolving Frameworks, [2] Don Roberts and Ralph Johnson suggest
 in the Tree Examples pattern that you should never write a software
 framework unless you have at least three applications that use it.  Part
 of the reasoning is that these applications help you come up with the
 proper abstractions for the framework.  If you think of NetBSD as a
 framework and the machine architectures and hardware as applications,
 then perhaps all of those applications can actually be a benefit to
 NetBSD and help it to have really nice abstractions and design.

Each new application that uses the framework adds drag, slowing the
framework's ability to evolve; while some of that drag is due to code,
a surprising amount is politics.
Each time the framework is changed there's probabilistic risk that an
application breaks.  The real problem, though, is when no one notices,
yet demands that the application be maintain continue loud and clear.

Here's a suggestion:

- as part of proposing that code be removed, provide supporting
evidence of when/why it broke

- and conversely, anyone suggesting that the bug is an easy fix, back
that up with a test platform so this can be confirmed

Based on experience, if there is evidence of long-broken code (as in
several major releases back) then this is just the start of the
problems.  If the functionality (not the code) really is worth
salvaging then that is something better to do _after_ the overhaul
that caused this to be flushed out.

Andrew

Who is appalled to discover that pc532 support has been removed!

Re: .eh_frame

[Andrew Cagney]

On 9 November 2014 17:12, Joerg Sonnenberger jo...@britannica.bec.de wrote:
  o .eh_frame in kernel is not used yet, and safely removed from /netbsd
 
  Please do not.

 o Is it correct that .eh_frame is not used by anything at all at the moment?

 gdb should in principle, haven't tried. libunwind is not hooked into ddb
 (yet).

Can you be more specific?

A remote debugger will call on either .debug_frame or .eh_frame when
generating a back-trace - what it uses depends on what it chooses to
look for first at each address. In fact, ignoring the potential for
bugs, you could:
- strip .eh_frame
- strip all debug info except .debug_frame
and still have good back-traces without weighing down the kernel's
text segment with .eh_frame info.

Andrew

Re: .eh_frame

[Andrew Cagney]

On 10 November 2014 18:21, Joerg Sonnenberger jo...@britannica.bec.de wrote:
 Consider x86_64 where you can't do reliable stack unwinding without also
 disabling -fomit-frame-pointer. The question is not about .debug_frame
 vs .eh_frame, you don't get the former at all without explicitly asking
 for debug data.

You've lost me.  I agree that without the help of CFI (call frame
information), amd64's stack is impenetrable goop.  That just means
that we need to ensure that there is cfi information on hand should
someone need it - in the .debug_frame section instead of .eh_frame.

(small technical nit, currently to get just the .debug_frame section,
gcc seems to require: -fno-unwind-tables
-fno-asynchronous-unwind-tables -g; which seems a tad excessive given
that gas's .cfi_sections pretty much does all we need)

As an aside, even with no CFI, it's still possible to get a feel for
what went down - just scan the stack for what look like code
addresses.

Re: .eh_frame

[Andrew Cagney]

On 10 November 2014 06:53, Masao Uebayashi uebay...@gmail.com wrote:

 So, does your kernel contain C++ code using exceptions, hand written
 eh-frame code, or an in kernel consumer? :-)

 According to joerg@, there is nothing (yet).

BTW, looks like arm (sys/arch/arm/include/profile.h) may contain
hand-written eh-frame code.  That shouldn't be a problem as, from gas:

7.11 `.cfi_sections SECTION_LIST'
=

`.cfi_sections' may be used to specify whether CFI directives should
emit `.eh_frame' section and/or `.debug_frame' section.  If
SECTION_LIST is `.eh_frame', `.eh_frame' is emitted, if SECTION_LIST is
`.debug_frame', `.debug_frame' is emitted.  To emit both use
`.eh_frame, .debug_frame'.  The default if this directive is not used
is `.cfi_sections .eh_frame'.

it is easy to change.

It's just a shame that more ports aren't exploiting this feature.
Makes unwinding less debugger-tool centric.

Re: .eh_frame

[Andrew Cagney]

You might find this useful:

http://lists.cs.uiuc.edu/pipermail/lldb-dev/2014-October/005546.html

Several notes:

- while eh_frame is meant to be the minimum information needed to
unwind function calls, the desire to unwind through C code and signals
from C++ means that most systems ship with the same detailed
information as found in .debug_frame

- assembler is often augmented with hand written eh-frame code so that
the unwinder's life is made easier (I should check but I suspect that
it doesn't duplicate the info to debug-frame so stripping it out will
make a debuggers life hard)

So, does your kernel contain C++ code using exceptions, hand written
eh-frame code, or an in kernel consumer? :-)
Another useful data point might be to post the numbers, how much space
does it take up?

PS: If it isn't a GNU extension (as in first implemented in GCC), I'd
love to know what it is.



On 9 November 2014 17:12, Joerg Sonnenberger jo...@britannica.bec.de wrote:
 On Mon, Nov 10, 2014 at 01:40:59AM +0900, Masao Uebayashi wrote:
 On Mon, Nov 10, 2014 at 1:24 AM, Joerg Sonnenberger
 jo...@britannica.bec.de wrote:
  On Sun, Nov 09, 2014 at 09:26:45PM +0900, Masao Uebayashi wrote:
  o .eh_frame is GNU extension debug info to unwind stack [1]
 
  No.

 Ian Lance Taylor said that it is similar to DWARF .debug_frame, but 
 different.

 Yes, but it is not a GNU extension.

  o .eh_frame in kernel is not used yet, and safely removed from /netbsd
 
  Please do not.

 o Is it correct that .eh_frame is not used by anything at all at the moment?

 gdb should in principle, haven't tried. libunwind is not hooked into ddb
 (yet).

 o Can users safely trim .eh_frame from her/his kernel?

 Yes, but it should not be the default behavior...

 Joerg