Re: gcc behavior of init priority of .ctors and .dtors section

[Zhenlei Huang]

> On May 17, 2024, at 2:26 AM, Konstantin Belousov  wrote:
> 
> On Thu, May 16, 2024 at 08:06:46PM +0800, Zhenlei Huang wrote:
>> Hi,
>> 
>> I'm recently working on https://reviews.freebsd.org/D45194 and got noticed
>> that gcc behaves weirdly.
>> 
>> A simple source file to demonstrate that.
>> 
>> ```
>> # cat ctors.c
>> 
>> #include 
>> 
>> __attribute__((constructor(101))) void init_101() { puts("init 1"); }
>> __attribute__((constructor(65535))) void init_65535() { puts("init 3"); }
>> __attribute__((constructor)) void init() { puts("init 4"); }
>> __attribute__((constructor(65535))) void init_65535_2() { puts("init 5"); }
>> __attribute__((constructor(65534))) void init_65534() { puts("init 2"); }
>> 
>> int main() { puts("main"); }
>> 
>> __attribute__((destructor(65534))) void fini_65534() { puts("fini 2"); }
>> __attribute__((destructor(65535))) void fini_65535() { puts("fini 3"); }
>> __attribute__((destructor)) void fini() { puts("fini 4"); }
>> __attribute__((destructor(65535))) void fini_65535_2() { puts("fini 5"); }
>> __attribute__((destructor(101))) void fini_101() { puts("fini 1"); }
>> 
>> # clang ctors.c && ./a.out
>> init 1
>> init 2
>> init 3
>> init 4
>> init 5
>> main
>> fini 5
>> fini 4
>> fini 3
>> fini 2
>> fini 1
>> ```
>> 
>> clang with the option -fno-use-init-array and run will produce the same 
>> result, which
>> is what I expected.
> Why do you add that switch?

gcc13 in ports is not configured with option --enable-initfini-array then it 
only produces .ctors / .dtors sections but
not .init_array / .fini_array sections. So I add that switch for clang to 
produce `.ctors` sections instead as
a baseline ( .ctors produced by clang indeed works as expected, the same with 
.init_array ).

> 
>> 
>> gcc13 from ports
>> ```
>> # gcc ctors.c && ./a.out
>> init 1
>> init 2
>> init 5
>> init 4
>> init 3
>> main
>> fini 3
>> fini 4
>> fini 5
>> fini 2
>> fini 1
>> ```
>> 
>> The above order is not expected. I think clang's one is correct.
>> 
>> Further hacking with readelf shows that clang produces the right order of
>> section .rela.ctors but gcc does not.
>> 
>> ```
>> # clang -fno-use-init-array -c ctors.c && readelf -r ctors.o | grep 
>> 'Relocation section with addend (.rela.ctors)' -A5 > clang.txt
>> # gcc -c ctors.c && readelf -r ctors.o | grep 'Relocation section with 
>> addend (.rela.ctors)' -A5 > gcc.txt
>> # diff clang.txt gcc.txt
>> 3,5c3,5
>> <  00080001 R_X86_64_64 0060 
>> init_65535_2 + 0
>> < 0008 00070001 R_X86_64_64 0040 init + 0
>> < 0010 00060001 R_X86_64_64 0020 init_65535 
>> + 0
>> ---
>>>  00060001 R_X86_64_64 0011 init_65535 + >>> 0
>>> 0008 00070001 R_X86_64_64 0022 init + 0
>>> 0010 00080001 R_X86_64_64 0033 init_65535_2 
>>> + 0
>> ```
>> 
>> The above show clearly gcc produces the wrong order of section `.rela.ctors`.
>> 
>> Is that expected behavior ?
>> 
>> I have not tried Linux version of gcc.
> Note that init array vs. init function behavior is encoded by a note added
> by crt1.o.  I suspect that the problem is that gcc port is built without
> --enable-initfini-array configure option.

gcc behavior of init priority of .ctors and .dtors section

[Zhenlei Huang]

Hi,

I'm recently working on https://reviews.freebsd.org/D45194 and got noticed
that gcc behaves weirdly.

A simple source file to demonstrate that.

```
# cat ctors.c

#include 

__attribute__((constructor(101))) void init_101() { puts("init 1"); }
__attribute__((constructor(65535))) void init_65535() { puts("init 3"); }
__attribute__((constructor)) void init() { puts("init 4"); }
__attribute__((constructor(65535))) void init_65535_2() { puts("init 5"); }
__attribute__((constructor(65534))) void init_65534() { puts("init 2"); }

int main() { puts("main"); }

__attribute__((destructor(65534))) void fini_65534() { puts("fini 2"); }
__attribute__((destructor(65535))) void fini_65535() { puts("fini 3"); }
__attribute__((destructor)) void fini() { puts("fini 4"); }
__attribute__((destructor(65535))) void fini_65535_2() { puts("fini 5"); }
__attribute__((destructor(101))) void fini_101() { puts("fini 1"); }

# clang ctors.c && ./a.out
init 1
init 2
init 3
init 4
init 5
main
fini 5
fini 4
fini 3
fini 2
fini 1
```

clang with the option -fno-use-init-array and run will produce the same result, 
which
is what I expected.

gcc13 from ports
```
# gcc ctors.c && ./a.out
init 1
init 2
init 5
init 4
init 3
main
fini 3
fini 4
fini 5
fini 2
fini 1
```

The above order is not expected. I think clang's one is correct.

Further hacking with readelf shows that clang produces the right order of
section .rela.ctors but gcc does not.

```
# clang -fno-use-init-array -c ctors.c && readelf -r ctors.o | grep 'Relocation 
section with addend (.rela.ctors)' -A5 > clang.txt
# gcc -c ctors.c && readelf -r ctors.o | grep 'Relocation section with addend 
(.rela.ctors)' -A5 > gcc.txt
# diff clang.txt gcc.txt
3,5c3,5
<  00080001 R_X86_64_64 0060 init_65535_2 + 0
< 0008 00070001 R_X86_64_64 0040 init + 0
< 0010 00060001 R_X86_64_64 0020 init_65535 + 0
---
>  00060001 R_X86_64_64 0011 init_65535 + 0
> 0008 00070001 R_X86_64_64 0022 init + 0
> 0010 00080001 R_X86_64_64 0033 init_65535_2 + > 0
```

The above show clearly gcc produces the wrong order of section `.rela.ctors`.

Is that expected behavior ?

I have not tried Linux version of gcc.


Best regards,
Zhenlei

Re: after commit 5e248c2, kernel module build is broken

[Zhenlei Huang]

> On Feb 25, 2024, at 11:38 PM, Michael Butler  
> wrote:
> 
> Building 
> /usr/obj/usr/src/amd64.amd64/sys/VM01-new/modules/usr/src/sys/modules/vmm/machine
> machine -> /usr/src/sys/amd64/include
> Building 
> /usr/obj/usr/src/amd64.amd64/sys/VM01-new/modules/usr/src/sys/modules/vmm/x86
> x86 -> /usr/src/sys/x86/include
> Building 
> /usr/obj/usr/src/amd64.amd64/sys/VM01-new/modules/usr/src/sys/modules/vmm/i386
> i386 -> /usr/src/sys/i386/include
> Building 
> /usr/obj/usr/src/amd64.amd64/sys/VM01-new/modules/usr/src/sys/modules/vmm/opt_acpi.h
> Building 
> /usr/obj/usr/src/amd64.amd64/sys/VM01-new/modules/usr/src/sys/modules/vmm/opt_ddb.h
> Building /usr/obj/usr/src/amd64.amd64/sys/VM01-new/cc.o
> /usr/src/sys/netinet/cc/cc.c:475:10: error: 6 enumeration values not handled 
> in switch: 'CC_ACK', 'CC_DUPACK', 'CC_PARTIALACK'... [-Werror,-Wswitch]
>  475 | switch (type) {
>  | ^~~~

Should be fixed by 8917131e00b0 (tcp: need default in switch statement for enum)

> 1 error generated.
> *** [cc.o] Error code 1
> 

Best regards,
Zhenlei

Re: loading vmm.ko crashes main-n267684-e4b646ce1610

[Zhenlei Huang]

> On Jan 24, 2024, at 8:47 AM, void  wrote:
> 
> Hello,
> 
> Loading vmm either via vmm_load=YES in /boot/loader.conf or
> manually via kldload vmm crashes the system to db> prompt
> as shown in the link below:
> 
> http://void.f-m.fm.user.fm/Screenshot_20240122_135539.png
> 
> Thing is, access is via a html5 console with no up/down scrolling
> or buffer. Nothing can be entered into the db> prompt. There's plenty
> more information when it crashes, but the output scrolls off the top.
> 
> The main-n267684 is running a modified GENERIC-NODEBUG
> kernel and has debugging turned off in /etc/src.conf 
> (it has WITHOUT_ASSERT_DEBUG= and WITHOUT_DEBUG_FILES= and 
> WITH_MALLOC_PRODUCTION=)
> 
> Apart from turning these on again and rebuilding GENERIC, what other things
> need to be set to get a usable coredump when this happens?

You may want to configure the dump device before get the coredump.
See  
https://docs.freebsd.org/en/books/developers-handbook/kerneldebug/#kerneldebug-obtain
 

 .

> 
> /var/crash has nothing in it.
> 
> thanks in advance for any assistance
> -- 
> 

Best regards,
Zhenlei

Re: route ipv6 errors on bootup in -current main-n267425-aa1223ac3afc on arm64

[Zhenlei Huang]

> On Jan 9, 2024, at 6:24 PM, void  wrote:
> 
> On Mon, Jan 08, 2024 at 01:07:30PM -0800, Enji Cooper wrote:
>> 
>> Was the kernel/utility built with IPv6? If not, that’s a general bug which 
>> should be filed (which can be easily checked/avoided using the FEATURES(9) 
>> subsystem)…
>> Cheers!
>> -Enji
> 
> world/kernel was built with WITHOUT_INET6= in /etc/src.conf
> 
> I made the problem go away with removing WITHOUT_INET6= and rebuilding.
> The system was installed by taking 
> FreeBSD-15.0-CURRENT-arm64-aarch64-RPI-20240104-8bf0882e186e-267378.img
> and dd-ing it to a usb3-connected hd.
> 
> Where can I read about features?

Features can be retrieved by `sysctl kern.features`.

As for INET6 it should be `kern.features.inet6` .

> 
> % man features
> No manual entry for "features"
> 
> it's not in apropos
> thanks,
> -- 
> 

Best regards,
Zhenlei

Re: IPFW/IPv6 problem with JAIL: JAIL cannot ping -6 host until host first pings jail (ipv6)

[Zhenlei Huang]

> On Jan 8, 2024, at 1:50 AM, FreeBSD User  wrote:
> 
> Hello,
> 
> I've got a problem with recent CURRENT, running vnet JAILs.
> FreeBSD 15.0-CURRENT #28 main-n267432-e5b33e6eef7: Sun Jan  7 13:18:15 CET 
> 2024 amd64
> 
> Main Host has IPFW configured and is open for services like OpenLDAP on 
> UDP/TCP and ICMP
> (ipfw is configured via rc.conf in this case, host is listening on both 
> protocol families
> IPv4 and IPv6). 
> 
> The host itself has openldap-server 2.6 as a service. The host's interface is 
> igb0 with
> assigned ULA. JAILs (around eight jails) are sharing their vnet interfaces 
> via a bridge with
> the same physical device as the host (igb0). After a while (the time elapsed 
> is unspecific)

How did you create your jails , are they vnet jails ? 

Is that bridge + epair ?

> the jail is unable to contact the host via IPv6: neither UDP, TCP nor ICMP 
> sent from the JAIL
> is reaching the host. IPv4 is working like a charme! No problems there.
> 
> When pinging the Jail from the main host via ping -6, the jail is responding! 
> After the first
> ping -6, the jail now is able to ping -6 the main host.
> 
> After a fresh reboot, the problem is not present and occurs after a while and 
> it seems to
> happen first to very active jails.
> 
> Kind regards,
> 
> oh
> 
> 
> -- 
> O. Hartmann
> 

Best regards,
Zhenlei

Re: How much survives an install/reboot cycle?

[Zhenlei Huang]

> On Nov 19, 2023, at 11:51 PM, bob prohaska  wrote:
> 
> How much of a running system's state survives a reboot? I used to think
> the answer was "nothing", but from time to time a second reboot behaves
> a  little differently from the previous one. 

Warner has a good description about that. I totally agree.

> 
> The most recent example was an update to bpf.c: Prior to the update an
> armv7 system had been inclined to drop ssh connections left up for days.
> After updating and running a build/install cycle the behavior persisted,
> but since a second reboot with no intentional changes it has stopped.

The most recent change to bpf.c is 7a974a649848 (bpf: Make dead_bpf_if const) .
It is not a functional change, and I do not think it will affect ssh.
There could be issues under the earth.

Anyway please do not hesitate to report if you get recovered by reverting 
7a974a649848.

> 
> I've not tampered with nextboot, so I don't think that's it. Maybe I'm
> just imagining imagining things 
> 
> Thanks for reading,
> 
> bob prohaska
> 
> 

Best regards,
Zhenlei

Re: Request for Testing: TCP RACK

[Zhenlei Huang]

> On Nov 19, 2023, at 2:35 AM, Zhenlei Huang  wrote:
> 
> 
> 
>> On Nov 16, 2023, at 5:13 PM, tue...@freebsd.org wrote:
>> 
>> Dear all,
>> 
>> recently the main branch was changed to build the TCP RACK stack
>> which is a loadable kernel module, by default:
>> https://cgit.FreeBSD.org/src/commit/?id=3a338c534154164504005beb00a3c6feb03756cc
>> 
>> As discussed on the bi-weekly transport call, it would be great if people
>> could test the RACK stack for their workload. Please report any problems to 
>> the
>> net@ mailing list or open an issue in the bug tracker and drop me a note via 
>> email.
>> This includes regressions in CPU usage, regressions in performance or any 
>> other
>> unexpected change you observe.
> 
> I see some performance regression with the rack stack.
> 
> This is iperf3 test on local host (bare metal, an old i5 2 cores 4 threads 
> MBP).
> 
> freebsd:  16.1 Gbits/sec
> rack: 12.3 Gbits/sec
> 
> The congestion control algorithm is cubic.
> 
> Note this is a quick test with DEBUG options enabled.
> 
> I'll try with no debug options and report later.

** UPDATE **

With no debug options:

freebsd:37.2 Gbits/sec
rack:   27.9 Gbits/sec

> 
>> 
>> You can load the kernel module using
>> kldload tcp_rack
>> 
>> You can make the RACK stack the default stack using
>> sysctl net.inet.tcp.functions_default=rack
>> 
>> Based on the feedback we get, the default stack might be switched to the
>> RACK stack.
>> 
>> Please let me know if you have any questions.
>> 
>> Best regards
>> Michael
>> 
>> 
>> 
> 
> Best regards,
> Zhenlei

Re: Request for Testing: TCP RACK

[Zhenlei Huang]

> On Nov 16, 2023, at 5:13 PM, tue...@freebsd.org wrote:
> 
> Dear all,
> 
> recently the main branch was changed to build the TCP RACK stack
> which is a loadable kernel module, by default:
> https://cgit.FreeBSD.org/src/commit/?id=3a338c534154164504005beb00a3c6feb03756cc
> 
> As discussed on the bi-weekly transport call, it would be great if people
> could test the RACK stack for their workload. Please report any problems to 
> the
> net@ mailing list or open an issue in the bug tracker and drop me a note via 
> email.
> This includes regressions in CPU usage, regressions in performance or any 
> other
> unexpected change you observe.

I see some performance regression with the rack stack.

This is iperf3 test on local host (bare metal, an old i5 2 cores 4 threads MBP).

freebsd:16.1 Gbits/sec
rack:   12.3 Gbits/sec

The congestion control algorithm is cubic.

Note this is a quick test with DEBUG options enabled.

I'll try with no debug options and report later.

> 
> You can load the kernel module using
> kldload tcp_rack
> 
> You can make the RACK stack the default stack using
> sysctl net.inet.tcp.functions_default=rack
> 
> Based on the feedback we get, the default stack might be switched to the
> RACK stack.
> 
> Please let me know if you have any questions.
> 
> Best regards
> Michael
> 
> 
> 

Best regards,
Zhenlei

Re: kldunload kernel: How should the kernel behave when it is requested to unload itself

[Zhenlei Huang]

> On Nov 10, 2023, at 1:28 AM, Tomoaki AOKI  wrote:
> 
> On Fri, 10 Nov 2023 00:10:13 +0800
> Zhenlei Huang  wrote:
> 
>> Hi,
>> 
>> This is *NOT* joking.
>> 
>> While working on https://reviews.freebsd.org/D42527 I realized the
>> module kernel also has userrefs, that is to say, userland can request
>> to unload kernel, aka `kldunload kernel`.
>> 
>> This is interesting. Well no doubt that the loader can unload kernel.
>> Then after the kernel is loaded and has been initialized (SYSINIT), how
>> should it behave when it get an unload request?
>> 
>> I'm proposing https://reviews.freebsd.org/D42530 to do not allow unloading
>> the kernel. It is by intuition.
>> 
>> What do you think ?
>> 
>> 
>> Best regards,
>> Zhenlei
> 
> Possibly too paranoid, but the summery on D42530 looks a bit confusing.
> Would better to be
> 
> 'The userland or kernel shall not unload the module "kernel".'
> 
> or
> 
> 'The userland or kernel shall not unload the "kernel" module.'
> 
> .
> 
> The original SUMMARY could be read as, in meaning, 'The userland or
> kernel shall not unload *.ko.'
> 
> *.ko is sometimes called as "kernel module", although it stants for
> "kernel object".

Thanks for point that out. I'll polish before committing it.

> 
> -- 
> Tomoaki AOKI

Re: kldunload kernel: How should the kernel behave when it is requested to unload itself

[Zhenlei Huang]

> On Nov 10, 2023, at 1:03 AM, Warner Losh  wrote:
> 
> Yea. Kexec is what you'd need to do to get a new kernel... and we don't 
> support kexec... so I agree this is good..

If we ever want to support kexec, a new kernel should be loaded into memory 
before the old one is unloaded.
Then probably a dedicated syscall is needed for that.

The current change D42530  is mainly to prevent userland from unloading the 
kernel via kldunload(2), so it should be OK.

> 
> Warner
> 
> On Thu, Nov 9, 2023, 9:34 AM Doug Rabson  <mailto:d...@rabson.org>> wrote:
> I think your intuition is correct - it never makes sense to unload the kernel 
> (IMO). I approved the review.
> 
> Doug.
> 
> 
> On Thu, 9 Nov 2023 at 16:10, Zhenlei Huang  <mailto:z...@freebsd.org>> wrote:
> Hi,
> 
> This is *NOT* joking.
> 
> While working on https://reviews.freebsd.org/D42527 
> <https://reviews.freebsd.org/D42527> I realized the
> module kernel also has userrefs, that is to say, userland can request
> to unload kernel, aka `kldunload kernel`.
> 
> This is interesting. Well no doubt that the loader can unload kernel.
> Then after the kernel is loaded and has been initialized (SYSINIT), how
> should it behave when it get an unload request?
> 
> I'm proposing https://reviews.freebsd.org/D42530 
> <https://reviews.freebsd.org/D42530> to do not allow unloading
> the kernel. It is by intuition.
> 
> What do you think ?
> 
> 
> Best regards,
> Zhenlei
> 

kldunload kernel: How should the kernel behave when it is requested to unload itself

[Zhenlei Huang]

Hi,

This is *NOT* joking.

While working on https://reviews.freebsd.org/D42527 I realized the
module kernel also has userrefs, that is to say, userland can request
to unload kernel, aka `kldunload kernel`.

This is interesting. Well no doubt that the loader can unload kernel.
Then after the kernel is loaded and has been initialized (SYSINIT), how
should it behave when it get an unload request?

I'm proposing https://reviews.freebsd.org/D42530 to do not allow unloading
the kernel. It is by intuition.

What do you think ?


Best regards,
Zhenlei

Re: KTLS thread on 14.0-RC3

[Zhenlei Huang]

> On Nov 1, 2023, at 8:37 AM, Rick Macklem  wrote:
> 
> On Tue, Oct 31, 2023 at 10:06 AM John Baldwin  <mailto:j...@freebsd.org>> wrote:
>> 
>> On 10/30/23 3:41 AM, Zhenlei Huang wrote:
>>> 
>>> 
>>>> On Oct 30, 2023, at 12:09 PM, Zhenlei Huang  wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Oct 29, 2023, at 5:43 PM, Gordon Bergling  wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> I am currently building a new system, which should be based on 
>>>>> 14.0-RELEASE.
>>>>> Therefor I am tracking releng/14.0 since its creation and updating it 
>>>>> currently
>>>>> via the usualy buildworld steps.
>>>>> 
>>>>> What I have noticed recently is, that the [KTLS] is missing. I have a 
>>>>> stable/13
>>>>> system which shows the [KTLS] thread and a very recent -CURRENT that also 
>>>>> shows
>>>>> the [KTLS] thread.
>>>>> 
>>>>> The stable/13 and releng/14.0 systems both use the GENERIC kernel, 
>>>>> without any
>>>>> custom modifications.
>>>>> 
>>>>> Loaded KLDs are also the same.
>>>>> 
>>>>> Did I miss something, or is there something in releng/14.0 missing, which
>>>>> is currenlty enabled in stable/13?
>>>> 
>>>> KTLS shall still work as intended, the creation of it threads is deferred.
>>>> 
>>>> See a72ee355646c (ktls: Defer creation of threads and zones until first 
>>>> use)
>>>>> Run ktls_init() when the first KTLS session is created rather than
>>>>> unconditionally during boot.  This avoids creating unused threads and
>>>>> allocating unused resources on systems which do not use KTLS.
>>>> 
>>>> ```
>>>> -SYSINIT(ktls, SI_SUB_SMP + 1, SI_ORDER_ANY, ktls_init, NULL);
>>>> ```
>>> 
>>> Seems 14.0 only create one KTLS thread.
>>> 
>>> IIRC 13.2 create one thread per core.
>> 
>> That part should not be different.  There should always be one thread per 
>> core.
> Just fyi, I see one thread/core.
> Did you happen to do something like "ps ax" instead of "ps axHl"?

Yes, I typed "ps auxx".  `ps axHl` is the right way to get kernel threads.
Sorry for the noise.

> 
> rick
> ps: I also see a reclaim_0 thread.
> 
>> 
>> --
>> John Baldwin

Re: KTLS thread on 14.0-RC3

[Zhenlei Huang]

> On Oct 30, 2023, at 12:09 PM, Zhenlei Huang  wrote:
> 
> 
> 
>> On Oct 29, 2023, at 5:43 PM, Gordon Bergling  wrote:
>> 
>> Hi,
>> 
>> I am currently building a new system, which should be based on 14.0-RELEASE.
>> Therefor I am tracking releng/14.0 since its creation and updating it 
>> currently
>> via the usualy buildworld steps.
>> 
>> What I have noticed recently is, that the [KTLS] is missing. I have a 
>> stable/13
>> system which shows the [KTLS] thread and a very recent -CURRENT that also 
>> shows
>> the [KTLS] thread.
>> 
>> The stable/13 and releng/14.0 systems both use the GENERIC kernel, without 
>> any
>> custom modifications.
>> 
>> Loaded KLDs are also the same.
>> 
>> Did I miss something, or is there something in releng/14.0 missing, which
>> is currenlty enabled in stable/13?
> 
> KTLS shall still work as intended, the creation of it threads is deferred.
> 
> See a72ee355646c (ktls: Defer creation of threads and zones until first use)
>> Run ktls_init() when the first KTLS session is created rather than
>> unconditionally during boot.  This avoids creating unused threads and
>> allocating unused resources on systems which do not use KTLS.
> 
> ```
> -SYSINIT(ktls, SI_SUB_SMP + 1, SI_ORDER_ANY, ktls_init, NULL);
> ```

Seems 14.0 only create one KTLS thread.

IIRC 13.2 create one thread per core.

> 
>> 
>> Any help for getting an insight on this would be much appreciated.
>> 
>> --Gordon
> 
> 
> Best regards,
> Zhenlei

Re: KTLS thread on 14.0-RC3

[Zhenlei Huang]

> On Oct 29, 2023, at 5:43 PM, Gordon Bergling  wrote:
> 
> Hi,
> 
> I am currently building a new system, which should be based on 14.0-RELEASE.
> Therefor I am tracking releng/14.0 since its creation and updating it 
> currently
> via the usualy buildworld steps.
> 
> What I have noticed recently is, that the [KTLS] is missing. I have a 
> stable/13
> system which shows the [KTLS] thread and a very recent -CURRENT that also 
> shows
> the [KTLS] thread.
> 
> The stable/13 and releng/14.0 systems both use the GENERIC kernel, without any
> custom modifications.
> 
> Loaded KLDs are also the same.
> 
> Did I miss something, or is there something in releng/14.0 missing, which
> is currenlty enabled in stable/13?

KTLS shall still work as intended, the creation of it threads is deferred.

See a72ee355646c (ktls: Defer creation of threads and zones until first use)
> Run ktls_init() when the first KTLS session is created rather than
> unconditionally during boot.  This avoids creating unused threads and
> allocating unused resources on systems which do not use KTLS.

```
-SYSINIT(ktls, SI_SUB_SMP + 1, SI_ORDER_ANY, ktls_init, NULL);
```

> 
> Any help for getting an insight on this would be much appreciated.
> 
> --Gordon


Best regards,
Zhenlei

Re: atrtc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 14

[Zhenlei Huang]

> On Oct 29, 2023, at 1:22 AM, Warner Losh  wrote:
> 
> 
> 
> On Sat, Oct 28, 2023, 11:04 AM Zhenlei Huang  <mailto:z...@freebsd.org>> wrote:
> Hi Warner,
> 
> I see this from boot log of 14.0-RC3:
> 
> > FreeBSD 14.0-RC3 #0 releng/14.0-n265368-c6cfdc130554: Fri Oct 27 05:57:28 
> > UTC 2023
> > ...
> > atrtc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 14.
> 
> I guess atrtc(4) is still supported by FreeBSD 14.
> So this should be bumped to 15 at least. Am I right ?
> 
> Yes. Someone, maybe you, submitted a pr to do that.

Posted to https://reviews.freebsd.org/D42387 
<https://reviews.freebsd.org/D42387> .


> 
> Warner
> 
> Best regards,
> Zhenlei
> 

atrtc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 14

[Zhenlei Huang]

Hi Warner,

I see this from boot log of 14.0-RC3:

> FreeBSD 14.0-RC3 #0 releng/14.0-n265368-c6cfdc130554: Fri Oct 27 05:57:28 UTC 
> 2023
> ...
> atrtc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 14.

I guess atrtc(4) is still supported by FreeBSD 14.
So this should be bumped to 15 at least. Am I right ?


Best regards,
Zhenlei

Re: git: 9d6eb30abb29 - stable/12 - x86: Add sysctl flag CTLFLAG_TUN to loader tunables

[Zhenlei Huang]

> On Oct 12, 2023, at 5:40 PM, Zhenlei Huang  wrote:
> 
> 
> 
>> On Oct 12, 2023, at 3:37 PM, Dewayne Geraghty  
>> wrote:
>> 
>> Thank-you so much for attending to AND correcting some of the weird 
>> anomalies.  Perhaps one day we can rely upon sysctl -aT and sysctl -aW to be 
>> meaningful again.
>> On 12.4S, 
>> A=`sysctl -a|wc -l`; T=`sysctl -aT|wc -l`; W=`sysctl -aW|wc -l`; O=`sysctl 
>> -ao|wc -l`;printf "Loadable=%d\tWriteable=%d\tStatistical=%d\tOpaque=%d\n" 
>> $T $W $(($A - $T - $W)) $(($O - $A))
>> Loadable=533Writeable=1347  Statistical=2560Opaque=50
>> 
>> Kind regards, Dewayne.
>> PS I'll refrain from mentioning the need for meaningful descriptions that 
>> provide a sense of how something can be "tuned".
>> 
>> 
> 
> 
> There're really too too many tunables to check, I'm currently mainly focusing 
> on platform independent part.

Done!

> 
> The next should be tier 1 , then tier 2.

Done!

Also note kib@ pointed out that there're some tunables for field debugging 
only, not much useful to expose them as sysctl MIBs. 

> 
> There're also many tunables in device drivers, there will be trouble to check 
> and fix them all as I believe a runtime test is mandatory 
> but that is basically impossible. Hope there're only little to be fixed.

I'm going to fix bugs related to net stack, and these are left unchanged.  I'd 
glad to do if they are strongly demanded.

> 
> Best regards,
> Zhenlei
> 

Re: Question about KBI change / new feature

[Zhenlei Huang]

> On Aug 23, 2023, at 1:06 AM, Warner Losh  wrote:
> 
> 
> 
> On Mon, Aug 21, 2023 at 9:42 AM Zhenlei Huang  <mailto:z...@freebsd.org>> wrote:
> Hi,
> 
> The https://www.freebsd.org/releases/14.0R/schedule/ 
> <https://www.freebsd.org/releases/14.0R/schedule/> says CURRENT/14 's KBI is 
> froze
> and new features should be avoided.
> 
> I'm working on https://reviews.freebsd.org/D39638 
> <https://reviews.freebsd.org/D39638> (sysctl(9): Enable vnet sysctl variables 
> be loader tunable)
> and I think it is new feature, but not quite sure whether the KBI changed.
> 
> So,
> 
> 1. Is it a KBI change ?
> 
> IMHO, It's a KPI change, not a KBI breakage. So from that perspective, it's 
> OK.

Thanks for the explanation !

>  
> 2. It is a simple change ( while so far as I know currently only tested by 
> myself on x86 and qemu riscv ), can
> it catch up with 14 ?
> 
> That I'm less sure of. I think it's good, but I'm gun shy about approving / 
> committing vnet things. The review suggests,
> though, there's at least some consensus for having this in the tree.

I always hesitate to PING someone to review ;)

Well I'm going to prepare to commit some of the stack, D41525, D39638, D39852, 
D39866, if no objections.

As for D40127, I have mixed filling about it. It might be too complex (for a 
simple function).
I wonder if we can have per-vnet `loader tunnable` to archive the same goal.

Best regards,
Zhenlei

Question about KBI change / new feature

[Zhenlei Huang]

Hi,

The https://www.freebsd.org/releases/14.0R/schedule/ says CURRENT/14 's KBI is 
froze
and new features should be avoided.

I'm working on https://reviews.freebsd.org/D39638 (sysctl(9): Enable vnet 
sysctl variables be loader tunable)
and I think it is new feature, but not quite sure whether the KBI changed.

So,

1. Is it a KBI change ?

2. It is a simple change ( while so far as I know currently only tested by 
myself on x86 and qemu riscv ), can
it catch up with 14 ?


Best regards,
Zhenlei

Re: Link modules to DYN type

[Zhenlei Huang]

> On Apr 28, 2023, at 12:32 AM, Zhenlei Huang  wrote:
> 
> 
> 
>> On Apr 26, 2023, at 7:12 PM, Konstantin Belousov > <mailto:kostik...@gmail.com>> wrote:
>> 
>> On Wed, Apr 26, 2023 at 12:55:02PM +0200, Hans Petter Selasky wrote:
>>> On 4/26/23 12:36, Zhenlei Huang wrote:
>>>> Hi,
>>>> 
>>>> I'm recently working on https://reviews.freebsd.org/D39638 
>>>> <https://reviews.freebsd.org/D39638> (sysctl(9): Enable vnet sysctl 
>>>> variables be loader tunable),
>>>> the changes to `sys/kern/link_elf_obj.c` are runtime tested, but not those 
>>>> to `sys/kern/link_elf.c` .
>>>> 
>>>> After some hacking I realized that `link_elf.c` is for EXEC (Executable 
>>>> file) or DYN (Shared object file), and `link_elf_obj.c` is
>>>> for REL (Relocatable file).
>>>> 
>>>> ```
>>>> /* link_elf.c */
>>>> static int
>>>> link_elf_load_file(linker_class_t cls, const char* filename,
>>>> linker_file_t* result)
>>>> {
>>>> ...
>>>>if (hdr->e_type != ET_EXEC && hdr->e_type != ET_DYN) {
>>>>error = ENOSYS;
>>>>goto out;
>>>>}
>>>> ...
>>>> }
>>>> 
>>>> 
>>>> /* link_elf_obj.c */
>>>> static int
>>>> link_elf_load_file(linker_class_t cls, const char *filename,
>>>> linker_file_t *result)
>>>> {
>>>> ...
>>>>if (hdr->e_type != ET_REL) {
>>>>error = ENOSYS;
>>>>goto out;
>>>>}
>>>> ...
>>>> }
>>>> ```
>>>> 
>>>> Run the following snip:
>>>> ```
>>>> # find /boot/kernel -type f -name "*.ko" -exec readelf -h {} \; | grep Type
>>>> ```
>>>> shows that all the kernel modules' types are `REL (Relocatable file)`.
>>>> 
>>>> I guess if some module such as if_bridge is linked to DYN type, then I can 
>>>> do runtime for the changes to `sys/kern/link_elf.c`.
>>>> 
>>>> I'm not familiar with elf and linkers, is that ( compile module and link 
>>>> it to DYN type ) possible ?
>> 
>> Module file type (shared object vs. object file) depends on architecture.
>> For amd64 modules are objects, while kernel is shared library.
>> For arm64 (and all other arches, I believe) modules and kernels are shared
>> libraries.
>> 
>> I think you can link amd64 module as shared object, but this require enough
>> hacking of the build infrastructure.  At least I am not aware of a simple
>> knob to switch the produced type.
> 
> I did some hack on `sys/conf/kmod.mk` and finally produced DYN kernel modules.
> The good news is I do some basic sysctl tests, but the bad news is the module 
> does not function correctly.
> 
> For exampe the if_disc.c
> 
> ```
> static void
> vnet_disc_init(const void *unused __unused)
> {
>   /* Reference V_disc_cloner will immediately trigger page fault panic */
>   V_disc_cloner = if_clone_simple(discname, disc_clone_create,
>   disc_clone_destroy, 0);
> }
> VNET_SYSINIT(vnet_disc_init, SI_SUB_PSEUDO, SI_ORDER_ANY,
> vnet_disc_init, NULL);
> ```
> 
> I suspect the relocation is not done correctly for  DYN elf kmod on amd64.
> 
> My local patch to kmod.mk:
> 
> ```
> diff --git a/sys/conf/kmod.mk b/sys/conf/kmod.mk
> index 134b150af1d9..1fc5386204a5 100644
> --- a/sys/conf/kmod.mk
> +++ b/sys/conf/kmod.mk
> @@ -84,6 +84,7 @@ __KLD_SHARED=yes
>  .else
>  __KLD_SHARED=no
>  .endif
> +__KLD_SHARED=yes
>  
>  .if !empty(CFLAGS:M-O[23s]) && empty(CFLAGS:M-fno-strict-aliasing)
>  CFLAGS+=   -fno-strict-aliasing
> @@ -167,6 +168,7 @@ CFLAGS+=-fno-omit-frame-pointer 
> -mno-omit-leaf-frame-pointer
>  ${MACHINE_CPUARCH} == "powerpc"
>  CFLAGS+=   -fPIC
>  .endif
> +CFLAGS+=   -fPIC
>  
>  # Temporary workaround for PR 196407, which contains the fascinating details.
>  # Don't allow clang to use fpu instructions or registers in kernel modules.
> ```
> 
> 
> As for https://reviews.freebsd.org/D39638 
> <https://reviews.freebsd.org/D39638>, for other platform such as arm, I think 
> the `link_elf_propagate_vnets()` should work if `parse_vnet()` works.
> 
> I'll appreciate if someone can test it on platforms those have DYN type 
> kernel modules. 

Good news on this!

I've managed to test the change for

Re: Link modules to DYN type

[Zhenlei Huang]

> On Apr 26, 2023, at 7:12 PM, Konstantin Belousov  wrote:
> 
> On Wed, Apr 26, 2023 at 12:55:02PM +0200, Hans Petter Selasky wrote:
>> On 4/26/23 12:36, Zhenlei Huang wrote:
>>> Hi,
>>> 
>>> I'm recently working on https://reviews.freebsd.org/D39638 (sysctl(9): 
>>> Enable vnet sysctl variables be loader tunable),
>>> the changes to `sys/kern/link_elf_obj.c` are runtime tested, but not those 
>>> to `sys/kern/link_elf.c` .
>>> 
>>> After some hacking I realized that `link_elf.c` is for EXEC (Executable 
>>> file) or DYN (Shared object file), and `link_elf_obj.c` is
>>> for REL (Relocatable file).
>>> 
>>> ```
>>> /* link_elf.c */
>>> static int
>>> link_elf_load_file(linker_class_t cls, const char* filename,
>>> linker_file_t* result)
>>> {
>>> ...
>>> if (hdr->e_type != ET_EXEC && hdr->e_type != ET_DYN) {
>>> error = ENOSYS;
>>> goto out;
>>> }
>>> ...
>>> }
>>> 
>>> 
>>> /* link_elf_obj.c */
>>> static int
>>> link_elf_load_file(linker_class_t cls, const char *filename,
>>> linker_file_t *result)
>>> {
>>> ...
>>> if (hdr->e_type != ET_REL) {
>>> error = ENOSYS;
>>> goto out;
>>> }
>>> ...
>>> }
>>> ```
>>> 
>>> Run the following snip:
>>> ```
>>> # find /boot/kernel -type f -name "*.ko" -exec readelf -h {} \; | grep Type
>>> ```
>>> shows that all the kernel modules' types are `REL (Relocatable file)`.
>>> 
>>> I guess if some module such as if_bridge is linked to DYN type, then I can 
>>> do runtime for the changes to `sys/kern/link_elf.c`.
>>> 
>>> I'm not familiar with elf and linkers, is that ( compile module and link it 
>>> to DYN type ) possible ?
> 
> Module file type (shared object vs. object file) depends on architecture.
> For amd64 modules are objects, while kernel is shared library.
> For arm64 (and all other arches, I believe) modules and kernels are shared
> libraries.
> 
> I think you can link amd64 module as shared object, but this require enough
> hacking of the build infrastructure.  At least I am not aware of a simple
> knob to switch the produced type.

I did some hack on `sys/conf/kmod.mk` and finally produced DYN kernel modules.
The good news is I do some basic sysctl tests, but the bad news is the module 
does not function correctly.

For exampe the if_disc.c

```
static void
vnet_disc_init(const void *unused __unused)
{
/* Reference V_disc_cloner will immediately trigger page fault panic */
V_disc_cloner = if_clone_simple(discname, disc_clone_create,
disc_clone_destroy, 0);
}
VNET_SYSINIT(vnet_disc_init, SI_SUB_PSEUDO, SI_ORDER_ANY,
vnet_disc_init, NULL);
```

I suspect the relocation is not done correctly for  DYN elf kmod on amd64.

My local patch to kmod.mk:

```
diff --git a/sys/conf/kmod.mk b/sys/conf/kmod.mk
index 134b150af1d9..1fc5386204a5 100644
--- a/sys/conf/kmod.mk
+++ b/sys/conf/kmod.mk
@@ -84,6 +84,7 @@ __KLD_SHARED=yes
 .else
 __KLD_SHARED=no
 .endif
+__KLD_SHARED=yes
 
 .if !empty(CFLAGS:M-O[23s]) && empty(CFLAGS:M-fno-strict-aliasing)
 CFLAGS+=   -fno-strict-aliasing
@@ -167,6 +168,7 @@ CFLAGS+=-fno-omit-frame-pointer 
-mno-omit-leaf-frame-pointer
 ${MACHINE_CPUARCH} == "powerpc"
 CFLAGS+=   -fPIC
 .endif
+CFLAGS+=   -fPIC
 
 # Temporary workaround for PR 196407, which contains the fascinating details.
 # Don't allow clang to use fpu instructions or registers in kernel modules.
```


As for https://reviews.freebsd.org/D39638 <https://reviews.freebsd.org/D39638>, 
for other platform such as arm, I think the `link_elf_propagate_vnets()` should 
work if `parse_vnet()` works.

I'll appreciate if someone can test it on platforms those have DYN type kernel 
modules. 

> 
> 
>>> 
>> 
>> Hi,
>> 
>> I don't have an answer for you either, but I have seen in the past, loading
>> kernel modules behaves a bit like libraries, in the following regard:
>> 
>> If two kernel modules define the same global symbol, then no warning is
>> given and the first loaded symbol definition (I think) is used to resolve
>> that symbol for all kernel modules, regardless of the prototype. Probably we
>> should not allow this. That's why building LINT is a good thing, to avoid
>> this issue.
> No, in-kernel linker does not behave this way.
> Modules need to contain explicit reference to all modules they depend upon,
> using the MODULE_DEPE

Link modules to DYN type

[Zhenlei Huang]

Hi,

I'm recently working on https://reviews.freebsd.org/D39638 (sysctl(9): Enable 
vnet sysctl variables be loader tunable),
the changes to `sys/kern/link_elf_obj.c` are runtime tested, but not those to 
`sys/kern/link_elf.c` .

After some hacking I realized that `link_elf.c` is for EXEC (Executable file) 
or DYN (Shared object file), and `link_elf_obj.c` is
for REL (Relocatable file).

```
/* link_elf.c */
static int
link_elf_load_file(linker_class_t cls, const char* filename,
linker_file_t* result)
{
...
if (hdr->e_type != ET_EXEC && hdr->e_type != ET_DYN) {
error = ENOSYS;
goto out;
}
...
}


/* link_elf_obj.c */
static int
link_elf_load_file(linker_class_t cls, const char *filename,
linker_file_t *result)
{
...
if (hdr->e_type != ET_REL) {
error = ENOSYS;
goto out;
}
...
}
```

Run the following snip:
```
# find /boot/kernel -type f -name "*.ko" -exec readelf -h {} \; | grep Type
```
shows that all the kernel modules' types are `REL (Relocatable file)`.

I guess if some module such as if_bridge is linked to DYN type, then I can do 
runtime for the changes to `sys/kern/link_elf.c`.

I'm not familiar with elf and linkers, is that ( compile module and link it to 
DYN type ) possible ?


Best regards,
Zhenlei

Re: Is it valid to combine CTLFLAG_TUN with CTLFLAG_VNET ?

[Zhenlei Huang]

> On Apr 6, 2023, at 3:56 AM, Hans Petter Selasky  wrote:
> 
> On 4/5/23 21:44, Hans Petter Selasky wrote:
>> On 4/5/23 20:23, Gleb Smirnoff wrote:
>>> What if we remove the CTLFLAG_VNET check from the code you posted above?
>>> I don't see anything going wrong, rather going right 
>>> 
>>> CTLFLAG_VNET will not mask away CTLFLAG_TUN.
>> Hi Gleb,
>> It's possible to bypass that check, but some work needs to be done first. 
>> Then all jails created, will also start from those sysctl tunable values.
>> The problem is, where does the VNET base pointer come from?
>> Especially those static sysctl's. You would need to make some design there I 
>> guess and look at the SYSINIT() order. When are SYSINIT's filled with 
>> tunable data's. And when is the default VNET created.
>> Because the data pointer passed to the register sysctl function is simply an 
>> offset pointer into a malloc'ed structure.
>> --HPS
> 
> Hi Zhenlei,
> 
> Feel free to work on this, and add me as a reviewer and complete phase two of:
> 
>> commit 3da1cf1e88f8448bb10c5f778ab56ff65c7a6938
>> Author: Hans Petter Selasky 
>> Date:   Fri Jun 27 16:33:43 2014 +
>>Extend the meaning of the CTLFLAG_TUN flag to automatically check if
>>there is an environment variable which shall initialize the SYSCTL
>>during early boot. This works for all SYSCTL types both statically and
>>dynamically created ones, except for the SYSCTL NODE type and SYSCTLs
>>which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added to
> 
> --HPS

Posted to https://reviews.freebsd.org/D39638 
 

CC freebsd-current if some people are interested in the fix.

Best regards,
Zhenlei

Re: Is it valid to combine CTLFLAG_TUN with CTLFLAG_VNET ?

[Zhenlei Huang]

> On Apr 6, 2023, at 3:56 AM, Hans Petter Selasky  wrote:
> 
> On 4/5/23 21:44, Hans Petter Selasky wrote:
>> On 4/5/23 20:23, Gleb Smirnoff wrote:
>>> What if we remove the CTLFLAG_VNET check from the code you posted above?
>>> I don't see anything going wrong, rather going right 
>>> 
>>> CTLFLAG_VNET will not mask away CTLFLAG_TUN.
>> Hi Gleb,
>> It's possible to bypass that check, but some work needs to be done first. 
>> Then all jails created, will also start from those sysctl tunable values.
>> The problem is, where does the VNET base pointer come from?
>> Especially those static sysctl's. You would need to make some design there I 
>> guess and look at the SYSINIT() order. When are SYSINIT's filled with 
>> tunable data's. And when is the default VNET created.
>> Because the data pointer passed to the register sysctl function is simply an 
>> offset pointer into a malloc'ed structure.
>> --HPS
> 
> Hi Zhenlei,
> 
> Feel free to work on this, and add me as a reviewer and complete phase two of:
> 
>> commit 3da1cf1e88f8448bb10c5f778ab56ff65c7a6938
>> Author: Hans Petter Selasky 
>> Date:   Fri Jun 27 16:33:43 2014 +
>>Extend the meaning of the CTLFLAG_TUN flag to automatically check if
>>there is an environment variable which shall initialize the SYSCTL
>>during early boot. This works for all SYSCTL types both statically and
>>dynamically created ones, except for the SYSCTL NODE type and SYSCTLs
>>which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added to
> 

I'd like to do some refactoring firstly, so that I can focus on CTLFLAG_VNET ;)

> --HPS

Best regards,
Zhenlei

Re: IFF_KNOWSEPOCH -> IFF_NEEDSEPOCH

[Zhenlei Huang]

> On Apr 7, 2023, at 2:34 AM, Gleb Smirnoff  wrote:
> 
>  Hi,
> 
> recently we had several drivers marked with IFF_KNOWSEPOCH
> which reminded me that this flag was supposed to be temporary.
> 
> Here is the change that introduced it e87c4940156. It was
> caused by several drivers sending packets from non-interrupt
> context and thus triggering NET_EPOCH_ASSERT(). It was about
> 10 - 20 drivers having this problem initially and reduced down
> to a few after 4426b2e64bd. We had a pretty heated dicussion
> back then and I apologize for that.
> 
> May I suggest before entering FreeBSD 14.0-RELEASE cycle we
> will get back to what was there before e87c4940156?

It sounds good if only a few drivers need IFF_NEEDSEPOCH.

> 
> To avoid the driver fallout that we used to have back in
> early 2020, here is the plan. In ether_input() where we
> now conditionally on the IFF_KNOWSEPOCH flag enter/exit the
> epoch with INVARIANTS we will also conditionally enter/exit
> in case we are supposed to be in the epoch wrt the flag, but
> we are not. We will also print a warning once, like "interface
> foo0 called if_input without epoch". This handling will be
> converted to normal assertion after a couple months.

Should also apply to infiniband_input().

BTW, is `in_epoch()` too heavy to replace flag IFF_NEEDSEPOCH
or IFF_KNOWSEPOCH ?

> 
> If everybody is fine with this suggestion I will post
> a review. Otherwise please express your concerns.

Good to make it moving forward ;)

Best regards,
Zhenlei

> 
> -- 
> Gleb Smirnoff

Re: NanoBSD: CURRENT unable to compile 13-STABLE : error: a function definition without a prototype is deprecated ... in C

[Zhenlei Huang]

> On Mar 8, 2023, at 10:26 PM, Ed Maste  wrote:
> 
> On Thu, 2 Mar 2023 at 05:14, Dimitry Andric  wrote:
>> 
>> WITHOUT_SYSTEM_COMPILER
>> Do not opportunistically skip building a cross-compiler during
>> the bootstrap phase of the build.  Normally, if the currently
> ...
>> I find the double negative phrasing "do not skip" always confusing. But
>> the logic is normally:
> 
> Yes, it's confusing -- perhaps we could rewrite it as something like
> "Build a cross-compiler during the build bootstrap phase, rather than
> opportunistically using the host's compiler."
> 

+1 for that.

Re: kernel compile fails if device bpf is not enabled

[Zhenlei Huang]

> On Feb 8, 2023, at 2:32 PM, Gary Jennejohn  wrote:
> 
> On Wed, 8 Feb 2023 12:49:46 +0800
> Zhenlei Huang mailto:z...@freebsd.org>> wrote:
> 
> Hi Zhenlei,
> 
>> Thanks for the report! Post the fix to Phabricator 
>> https://reviews.freebsd.org/D38432 <https://reviews.freebsd.org/D38432> 
>> <https://reviews.freebsd.org/D38432 <https://reviews.freebsd.org/D38432>> .
>> 
>> Best regards,
>> Zhenlei
>> 
> 
> Thanks!  I see it's been accepted.

Fixed in 9df6eeabb379 .

> 
>>> On Feb 7, 2023, at 5:08 PM, Gary Jennejohn  wrote:
>>> 
>>> I just saw this error today because I didn't have device bpf enabled in
>>> my kernel configuration file:
>>> 
>>> --
>>>>>> stage 3.1: building everything
>>> --
>>> linking kernel.full
>>> ld: error: undefined symbol: bpf_mtap_if
>>>>>> referenced by if.c:4724 (/usr/src/sys/net/if.c:4724)
>>>>>> if.o:(if_bpfmtap)
>>>>>> referenced by if_tuntap.c:1717 (/usr/src/sys/net/if_tuntap.c:1717)
>>>>>> if_tuntap.o:(tunread)
>>>>>> referenced by if_vlan.c:1292 (/usr/src/sys/net/if_vlan.c:1292)
>>>>>> if_vlan.o:(vlan_transmit)
>>> 
>>> ld: error: undefined symbol: bpf_mtap2_if
>>>>>> referenced by if_gif.c:323 (/usr/src/sys/net/if_gif.c:323)
>>>>>> if_gif.o:(gif_transmit)
>>>>>> referenced by if_tuntap.c:1816 (/usr/src/sys/net/if_tuntap.c:1816)
>>>>>> if_tuntap.o:(tunwrite)
>>> --- kernel.full ---
>>> *** [kernel.full] Error code 1
>>> 
>>> This happens because a dummy bpf_mtap_if() (called a NOP stub in bpf.c)
>>> isn't being defined if bpf is not enabled.
>>> 
>>> --
>>> Gary Jennejohn
>>> 
> 
> --
> Gary Jennejohn

Re: kernel compile fails if device bpf is not enabled

[Zhenlei Huang]

Hi Gary,

Thanks for the report! Post the fix to Phabricator 
https://reviews.freebsd.org/D38432  .

Best regards,
Zhenlei

> On Feb 7, 2023, at 5:08 PM, Gary Jennejohn  wrote:
> 
> I just saw this error today because I didn't have device bpf enabled in
> my kernel configuration file:
> 
> --
 stage 3.1: building everything
> --
> linking kernel.full
> ld: error: undefined symbol: bpf_mtap_if
 referenced by if.c:4724 (/usr/src/sys/net/if.c:4724)
  if.o:(if_bpfmtap)
 referenced by if_tuntap.c:1717 (/usr/src/sys/net/if_tuntap.c:1717)
  if_tuntap.o:(tunread)
 referenced by if_vlan.c:1292 (/usr/src/sys/net/if_vlan.c:1292)
  if_vlan.o:(vlan_transmit)
> 
> ld: error: undefined symbol: bpf_mtap2_if
 referenced by if_gif.c:323 (/usr/src/sys/net/if_gif.c:323)
  if_gif.o:(gif_transmit)
 referenced by if_tuntap.c:1816 (/usr/src/sys/net/if_tuntap.c:1816)
  if_tuntap.o:(tunwrite)
> --- kernel.full ---
> *** [kernel.full] Error code 1
> 
> This happens because a dummy bpf_mtap_if() (called a NOP stub in bpf.c)
> isn't being defined if bpf is not enabled.
> 
> --
> Gary Jennejohn
> 

Re: DESPARATE: How to stop FreeBSD form sleeping / disable ACPI? (on FreeBSD14 CURRENT)

[Zhenlei Huang]

> On Nov 11, 2022, at 4:29 AM,  
>  wrote:
> 
> I am still desperately trying to stop FreeBSD from sleeping, but I simply do 
> not manage. 
>  
> It is really very annoying that I have to restart the machine every 10 
> minutes, when I am working via SSH.

I think you will need to find the event triggering sleep (ACPI s1 / s3) every 
10 minutes.

> So if any one has a solution, it would be very much appreciated!
>  
> It should ….. be possible to kill / stop ACPI some how 
> If absolutely not possible in the actual build , a cron job restarting the 
> timer every 5 minutes perhaps !!???
>  
> It is possible perhaps … that GNOME is initiating this, despite that the GUI 
> powersetting is screenblank “NEVER”.  

Probably. (or some other components of the GNOME)

I've dozens of VMs / baremetal machines used as servers and routers. None of 
them sleep (without explicitly means such as acpiconf).
I've not use FreeBSD as desktop since about ten year ago.

> Whatever is causing the problem, the settings should be such that ^no 
> whatever program^ should not be capable to initiate the sleepmode. 
>  
>  
> Louis
> 
> I need to disable acpi and the indicated method for that is to add 
> ^hint.acpi.0.disabled="1"^ in /boot/loader.conf .
> However that crashes my system !! 
> Not only that, to make it work again I have to edit loader.conf on a system 
> which does ^not start^.  
>  
> After a lot of searching Internet came to the help with, I could start the 
> system again:
> 1. Select 3. Escape to loader prompt at the splash screen
> 2. Type set hint.acpi.0.disabled="0" on the loader prompt
> 3. Then type boot on the loader prompt
> edit the loader.conf
> Very very glad with that fix however
>  
> However the problem is still there, no idea how to prevent the system from 
> going to sleep (after about 10 minutes).
> No idea how to change those 10 minutes to a much longer time as well  
>  
> Note that I have gnome as gui and use the system more or less as server and 
> manage the machine partly local via the GUI and partly remote via SSH.

I think you can disable GUI / GNOME completely and try again.

>  
> Related to GNOME I did try ^gsettings set 
> org.gnome.settings-daemon.plugins.power sleep-inactive-ac-timeout 0^, however 
> that did not solve the problem as well.
>  
> In the end there seems to two problems
> a) A BSD-issue ACPI-turn off in the bootloader is crashing the system ! ! and 
> b) a GNOME issue (switching the system off during user inactivity, which is 
> bullshit for a server / for ssh-login / with multiple users).
> What IMHO apart from the screen lock, this is not a GNOME task but an OS  
> function to be configured by the system administrator.
>  
> A third problem, not to be addressed here, is that recovery from sleep mode 
> does not work on my system as well (even not S1).
>  
> Most important for the moment is that the system keeps running / is not going 
> down after x-time ! 
>  
> Louis

Best regards,
Zhenlei

Re: linux debian jail - network problems

[Zhenlei Huang]

> On Mar 1, 2022, at 6:42 PM, Sami Halabi  wrote:
> 
> How can I see the netlink wip status ?

Sorry it is not currently public visible. FreeBSD's Phabricator is a tool that 
is development focused.
If you're interested in it, please CC the author Alexander V. Chernikov .

> 
> בתאריך יום ו׳, 25 בפבר׳ 2022, 08:34, מאת Sami Halabi ‏ <mailto:sodyn...@gmail.com>>:
> Hi,
> Thank you for your response.. I wonder if Is it really only netlink problem?

Maybe is or not. I'm not familiar with Linux emulation. You can refer to 
1. https://docs.freebsd.org/en/articles/linux-emulation/ 
<https://docs.freebsd.org/en/articles/linux-emulation/>
2. https://wiki.freebsd.org/Linuxulator <https://wiki.freebsd.org/Linuxulator>
> Their are fee problems in the logs.. I dont kbow if they all related only to 
> netlink (prctl immutable for example).. I also saw oncompatibilities in 
> socket.c 
> 
> Btw: I tried to enter the link you sent and it asked for username and 
> password.. its not public review?
> 
> Sami
> 
> בתאריך יום ו׳, 25 בפבר׳ 2022, 04:18, מאת Zhenlei Huang ‏ <mailto:zlei.hu...@gmail.com>>:
> Hi,
> You can also track the WIP netlink feature, 
> https://reviews.freebsd.org/D33975 <https://reviews.freebsd.org/D33975>
> 
>> On Feb 25, 2022, at 4:05 AM, Sami Halabi > <mailto:sodyn...@gmail.com>> wrote:
>> 
>> Hi,
>> Added Current, maybe will be lucky ;)
>> 
>> Anyone have idea how approach and fix this?
>> 
>> Sami
>> 
>> בתאריך יום ג׳, 22 בפבר׳ 2022, 23:30, מאת Sami Halabi ‏> <mailto:sodyn...@gmail.com>>:
>> Hi all,
>> sorry for the cross post but I need help and I'm not sure where it hangs.
>> 
>> I create linux jail (debian bullseye) via cbsd.
>> the jail is being populated with the debian userland..
>> so far so good... services running (sshd) and I can login to the jail, I 
>> also can update packages and I can install apache httpd and all works fine 
>> (apt install or make from src).
>> I also manage to install packages even if their scripts depend on "ip" 
>> command that fails:
>> cbsd@j2> ip
>> Cannot open netlink socket: Address family not supported by protocol
>> 
>> ifconfig show empty interfaces:
>> cbsd@j2> ifconfig
>> eth0: flags=4163  mtu 1500
>> ether 00:50:56:0a:b3:a0  (Ethernet)
>> RX packets 139798314  bytes 12029597009 (11.2 GiB)
>> RX errors 0  dropped 0  overruns 0  frame 0
>> TX packets 26879143  bytes 34400160833 (32.0 GiB)
>> TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>> 
>> lo0: flags=4169  mtu 16384
>> loop  (Local Loopback)
>> RX packets 28548  bytes 160312960 (152.8 MiB)
>> RX errors 0  dropped 0  overruns 0  frame 0
>> TX packets 28548  bytes 160312960 (152.8 MiB)
>> TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>> 
>> I know linux emulation doesn't implement netlink.. so what I do is fake the 
>> response by replacing /bin/ip by a bash script that prints the correct IP 
>> and fakes some other (needed by packages i Installed):
>> #!/bin/bash
>> if [ "$1" = "-o" ]; then
>> echo "1: eth0 inet 192.168.1.2/24 <http://192.168.1.2/24> brd 192.168.1.255 
>> scope global eth0"
>> elif [ "$1" = "route" ]; then
>> if [ "$2" = "get" ]; then
>> echo "8.8.8.8 via  192.168.1.2   dev eth0  src  192.168.1.2  
>> "
>> else
>> echo "default via  192.168.1.2   dev eth0"
>> fi
>> else
>> echo "1: eth0:  mtu 1500 qdisc mq state UP 
>> qlen 1000"
>> echo "  inet  192.168.1.2  /24 brd  192.168.1.255 scope global eth0"
>> 
>> 
>> still ifconfig shows no IP... its time to say it a regular jail and *NOT* 
>> VNET.
>> 
>> *however* package that pull ips via libraries fail..
>> eg: installed bind916 (name) in the logs I see these errors (relevant only):
>> cbsd@j2> service named start
>> Starting domain name service...: namednamed: prctl(PR_SET_DUMPABLE) failed: 
>> Invalid argument
>> cbsd@j2>
>> 
>> 
>> log file shows:
>> 22-Feb-2022 23:11:58.705 general: notice: BIND 9 is maintained by Internet 
>> Systems Consortium,
>> 22-Feb-2022 23:11:58.705 general: notice: Inc. (ISC), a non-profit 501(c)(3) 
>> public-benefit
>> 22-Feb-2022 23:11:58.705 general: notice: corporation.  Support and training 
>> for BIND 9 are
>> 22-Feb-20