[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

Rodney W. Grimes  changed:

   What|Removed |Added

 CC|sta...@freebsd.org  |

--- Comment #15 from Rodney W. Grimes  ---
Please do not put bugs on stable@, current@, hackers@, etc

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #14 from Andrey V. Elsukov  ---
(In reply to Sergey Anokhin from comment #13)
> (In reply to Andrey V. Elsukov from comment #11)
> 
> I'd preferred to try to rebuild kernel if it's no difference between turning
> off VIMAGE from kernel config and applying patch because kernel building
> more faster then "world" building. As far as I understand, you are propose
> patch for "world" component, right?

No, the patch is for kernel.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #13 from Sergey Anokhin  ---
(In reply to Andrey V. Elsukov from comment #11)

I'd preferred to try to rebuild kernel if it's no difference between turning
off VIMAGE from kernel config and applying patch because kernel building more
faster then "world" building. As far as I understand, you are propose patch for
"world" component, right?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #12 from Sergey Anokhin  ---
(In reply to Andrey V. Elsukov from comment #9)

Sure, now I'm building kernel without VIMAGE. I'll let you know about testing
result

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #11 from Andrey V. Elsukov  ---
Created attachment 201968
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=201968=edit
Proposed patch

Also, you can test this patch instead, it should fix panic with VIMAGE option.
The problem is due to introduced deferred PCB destroying via epoch_call().
Since this code is executed from gtaskqueue, it has no VNET context.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #10 from Sergey Anokhin  ---
(In reply to Jan Bramkamp from comment #6)

Will it ok?

(pts/1)[root@server:~]# sysctl kern.maxssiz=1073741824
kern.maxssiz: 536870912 -> 1073741824
(pts/1)[root@server:~]# /usr/local/etc/rc.d/racoon onestart
Starting racoon.
(pts/1)[root@server:~]# /usr/local/etc/rc.d/racoon onestop
Stopping racoon.
Waiting for PIDS: 5662

kernel panic

btw, I've noticed that kernel panic during stopping racoon.

# kgdb kernel /var/crash/vmcore.last
GNU gdb (GDB) 8.2.1 [GDB v8.2.1 for FreeBSD]
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd12.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from kernel...Reading symbols from
/usr/obj/usr/src/amd64.amd64/sys/SERVER/kernel.debug...done.
done.

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 02
fault virtual address   = 0x28
fault code  = supervisor read data, page not present
instruction pointer = 0x20:0x80ecd31d
stack pointer   = 0x28:0xfe003fca7a40
frame pointer   = 0x28:0xfe003fca7a60
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process = 0 (softirq_2)
trap number = 12
panic: page fault
cpuid = 2
time = 1550009599
KDB: stack backtrace:
#0 0x80c531c7 at kdb_backtrace+0x67
#1 0x80c07143 at vpanic+0x1a3
#2 0x80c06f93 at panic+0x43
#3 0x8118d9ff at trap_fatal+0x35f
#4 0x8118da59 at trap_pfault+0x49
#5 0x8118d07e at trap+0x29e
#6 0x81168ac5 at calltrap+0x8
#7 0x80eca240 at ipsec_delete_pcbpolicy+0x20
#8 0x80dbaeec at in_pcbfree_deferred+0x6c
#9 0x80c4db1a at epoch_call_task+0x1ca
#10 0x80c51a54 at gtaskqueue_run_locked+0x144
#11 0x80c516b8 at gtaskqueue_thread_loop+0x98
#12 0x80bc6f23 at fork_exit+0x83
#13 0x81169abe at fork_trampoline+0xe
Uptime: 8m33s
Dumping 950 out of 8077 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91%

__curthread () at ./machine/pcpu.h:230
230 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n"
(OFFSETOF_CURTHREAD));
(kgdb) bt
#0  __curthread () at ./machine/pcpu.h:230
#1  doadump (textdump=) at /usr/src/sys/kern/kern_shutdown.c:366
#2  0x80c06d2b in kern_reboot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:446
#3  0x80c071a3 in vpanic (fmt=, ap=0xfe003fca7790)
at /usr/src/sys/kern/kern_shutdown.c:872
#4  0x80c06f93 in panic (fmt=) at
/usr/src/sys/kern/kern_shutdown.c:799
#5  0x8118d9ff in trap_fatal (frame=0xfe003fca7980, eva=40) at
/usr/src/sys/amd64/amd64/trap.c:929
#6  0x8118da59 in trap_pfault (frame=0xfe003fca7980, usermode=0) at
/usr/src/sys/amd64/amd64/trap.c:765
#7  0x8118d07e in trap (frame=0xfe003fca7980) at
/usr/src/sys/amd64/amd64/trap.c:441
#8  
#9  0x80ecd31d in key_freesp (spp=0xf80211241880) at
/usr/src/sys/netipsec/key.c:1199
#10 0x80eca240 in ipsec_delete_pcbpolicy (inp=0xf800151aa1e8) at
/usr/src/sys/netipsec/ipsec_pcb.c:176
#11 0x80dbaeec in in_pcbfree_deferred (ctx=0xf800151aa3c0) at
/usr/src/sys/netinet/in_pcb.c:1576
#12 0x80c4db1a in epoch_call_task (arg=) at
/usr/src/sys/kern/subr_epoch.c:507
#13 0x80c51a54 in gtaskqueue_run_locked (queue=0xf80003363c00) at
/usr/src/sys/kern/subr_gtaskqueue.c:376
#14 0x80c516b8 in gtaskqueue_thread_loop (arg=) at
/usr/src/sys/kern/subr_gtaskqueue.c:557
#15 0x80bc6f23 in fork_exit (callout=0x80c51620
, arg=0xfe00025f5038, frame=0xfe003fca7c00)
at /usr/src/sys/kern/kern_fork.c:1059
#16 
(kgdb) frame 9
#9  0x80ecd31d in key_freesp (spp=0xf80211241880) at
/usr/src/sys/netipsec/key.c:1199
1199KEYDBG(IPSEC_STAMP,
(kgdb)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #9 from Andrey V. Elsukov  ---
Can you try to remove `option VIMAGE` from your kernel config? It looks like
the problem is similar to the one described in
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235699

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #8 from Sergey Anokhin  ---
(In reply to Jan Bramkamp from comment #6)

Did you mean try to set kern.maxssiz into /boot/loader.conf?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #7 from Sergey Anokhin  ---
btw, perhaps it can be helpful: if port security/ipsec-tools was built with
default options (make rmconfig), so the bug doesn't reproduced

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

Jan Bramkamp  changed:

   What|Removed |Added

 CC||cr...@bultmann.eu

--- Comment #6 from Jan Bramkamp  ---
Can you try again with IPSEC_DEBUG and a doubled kernel stack size?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #5 from Andrey V. Elsukov  ---
(In reply to Sergey Anokhin from comment #4)
> (In reply to Andrey V. Elsukov from comment #3)
> 
> There is a mind that if turn off
> 
> options IPSEC_DEBUG
> 
> kernel panic will disappear

Disabling IPSEC_DEBUG also reduces the requirement to kernel stack size.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #4 from Sergey Anokhin  ---
(In reply to Andrey V. Elsukov from comment #3)

There is a mind that if turn off

options IPSEC_DEBUG

kernel panic will disappear

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #3 from Andrey V. Elsukov  ---
KEYDBG() macro executed only when net.key.debug is set to non-zero value. It
looks like your sysctl.conf didn't set it. Also, it looks impossible to get
page fault with fault address 0x28 in this line of code. I suspect, that you
have some sort of memory corruption. Not sure, is it hardware related or it is
overwritten by some code.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

--- Comment #2 from Sergey Anokhin  ---
(In reply to Andrey V. Elsukov from comment #1)

kernel config:

(pts/2)[root@server:~]# cat /usr/src/sys/amd64/conf/SERVER
#
# GENERIC -- Generic kernel configuration file for FreeBSD/amd64
#
# For more information on this file, please read the config(5) manual page,
# and/or the handbook section on Kernel Configuration Files:
#
#   
https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html
#
# The handbook is also available locally in /usr/share/doc/handbook
# if you've installed the doc distribution, otherwise always see the
# FreeBSD World Wide Web server (https://www.FreeBSD.org/) for the
# latest information.
#
# An exhaustive list of options and more detailed explanations of the
# device lines is also present in the ../../conf/NOTES and NOTES files.
# If you are in doubt as to the purpose or necessity of a line, check first
# in NOTES.
#
# $FreeBSD: stable/12/sys/amd64/conf/GENERIC 340695 2018-11-20 19:37:09Z
zeising $

cpu HAMMER
ident   SERVER

makeoptions DEBUG=-g# Build kernel with gdb(1) debug
symbols
makeoptions WITH_CTF=1  # Run ctfconvert(1) for DTrace support

options SCHED_ULE   # ULE scheduler
options NUMA# Non-Uniform Memory Architecture
support
options PREEMPTION  # Enable kernel thread preemption
options VIMAGE  # Subsystem virtualization, e.g. VNET
options INET# InterNETworking
options INET6   # IPv6 communications protocols
options IPSEC   # IP (v4/v6) security
options IPSEC_SUPPORT   # Allow kldload of ipsec and tcpmd5
options TCP_OFFLOAD # TCP offload
options TCP_BLACKBOX# Enhanced TCP event logging
options TCP_HHOOK   # hhook(9) framework for TCP
options TCP_RFC7413 # TCP Fast Open
options SCTP# Stream Control Transmission Protocol
options FFS # Berkeley Fast Filesystem
options SOFTUPDATES # Enable FFS soft updates support
options UFS_ACL # Support for access control lists
options UFS_DIRHASH # Improve performance on big
directories
options UFS_GJOURNAL# Enable gjournal-based UFS journaling
options QUOTA   # Enable disk quotas for UFS
options MD_ROOT # MD is a potential root device
options NFSCL   # Network Filesystem Client
options NFSD# Network Filesystem Server
options NFSLOCKD# Network Lock Manager
options NFS_ROOT# NFS usable as /, requires NFSCL
options MSDOSFS # MSDOS Filesystem
options CD9660  # ISO 9660 Filesystem
options PROCFS  # Process filesystem (requires
PSEUDOFS)
options PSEUDOFS# Pseudo-filesystem framework
options GEOM_RAID   # Soft RAID functionality.
options GEOM_LABEL  # Provides labelization
options EFIRT   # EFI Runtime Services support
options COMPAT_FREEBSD32# Compatible with i386 binaries
options COMPAT_FREEBSD4 # Compatible with FreeBSD4
options COMPAT_FREEBSD5 # Compatible with FreeBSD5
options COMPAT_FREEBSD6 # Compatible with FreeBSD6
options COMPAT_FREEBSD7 # Compatible with FreeBSD7
options COMPAT_FREEBSD9 # Compatible with FreeBSD9
options COMPAT_FREEBSD10# Compatible with FreeBSD10
options COMPAT_FREEBSD11# Compatible with FreeBSD11
options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI
options KTRACE  # ktrace(1) support
options STACK   # stack(9) support
options SYSVSHM # SYSV-style shared memory
options SYSVMSG # SYSV-style message queues
options SYSVSEM # SYSV-style semaphores
options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time
extensions
options PRINTF_BUFR_SIZE=128# Prevent printf output being
interspersed.
options KBD_INSTALL_CDEV# install a CDEV entry in /dev
options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(4)
options AUDIT   # Security event auditing
options CAPABILITY_MODE # Capsicum capability mode
options CAPABILITIES# Capsicum capabilities
options MAC # TrustedBSD MAC Framework
options KDTRACE_FRAME   # Ensure frames are 

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

Andrey V. Elsukov  changed:

   What|Removed |Added

 CC||a...@freebsd.org

--- Comment #1 from Andrey V. Elsukov  ---
(In reply to Sergey Anokhin from comment #0)
> I see kernel panic during racoon restart.
> 
> # uname -rv
> 12.0-STABLE FreeBSD 12.0-STABLE r343904 SERVER

Please, show the content of your kernel config and what sysctl variables do you
changed against default configuration.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[Bug 235684] security/ipsec-tools kernel panic

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235684

Sergey Anokhin  changed:

   What|Removed |Added

 CC||b...@freebsd.org,
   ||sta...@freebsd.org

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"