>Synopsis: assertion failures in relayd after config reload
>Category: system
>Environment:
System : OpenBSD 7.5
Details : OpenBSD 7.5 (GENERIC.MP) #82: Wed Mar 20 15:48:40 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
Architecture: OpenBSD.amd64
Machine : amd64
>Description:
When running relayd on FreeBSD, one user reports that relayd
occasionally
exits with an internal error, typically "pfe: pfe_dispatch_hce:
desynchronized"
or "relay: relay_dispatch_pfe: invalid host id". The error occurs when
relayd's configuration file is reloaded.
I reproduced the problem on OpenBSD 7.5, see below for a simple
reproducer.
In particular, there appears to be a race condition involving three
processes:
the PFE, the HCE and the parent process. The race goes something like
this:
1. The parent process receives SIGHUP, which causes IMSG_CTL_RESET
messages
to be sent to the PFE and HCE.
2. The HCE sends some IMSG_HOST_STATUS messages to the PFE, following a
health
check. These messages include a count of the number of checks since
the
last reset. The PFE keeps track of this count; if it receives a
message
with a different count from what it expects, it'll exit.
3. The PFE receives IMSG_CTL_RESET and purges all of its state. Then it
reloads the configuration and repopulates it.
4. The PFE receives some of the IMSG_HOST_STATUS messages sent in step
2.
The check counts contained in those messages are stale, causing
errors.
This occurs only rarely in production, but it's easy to reproduce by
spamming relayd with poll and reload commands in parallel, see below.
>How-To-Repeat:
1. Set up a web server, in this example it's at 127.0.0.1:80.
2. Configure relayd using the following configuration:
---
table <test> {
127.0.0.1
}
redirect "test" {
listen on 127.0.0.1 tcp port 8080
forward to <test> port 80 mode roundrobin check http "/" host acs-areq code
200
session timeout 600
}
---
3. Start relayd. Let the following two loops run in parallel:
while true; do kill -HUP $(pgrep relayd); sleep 0.01; done
while true; do relayctl poll; sleep 0.01; done
4. Observe relayd exiting with an error.
>Fix:
I don't have a patch for this yet. I'm happy to work on it, especially
given some suggestions as to what shape the solution should have.
dmesg:
OpenBSD 7.5 (GENERIC.MP) #82: Wed Mar 20 15:48:40 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 489816064 (467MB)
avail mem = 454209536 (433MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x1fbcf000 (11 entries)
bios0: vendor BHYVE version "14.0" date 10/17/2021
bios0: FreeBSD BHYVE
efi0 at bios0: UEFI 2.7
efi0: BHYVE rev 0x10000
acpi0 at bios0: ACPI 5.1
acpi0: sleep states S5
acpi0: tables DSDT FACP APIC HPET MCFG SPCR BGRT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Ryzen 9 7950X3D 16-Core Processor, 4197.26 MHz, 19-61-02
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,LONG,LAHF,CMPLEG,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,SKINIT,TCE,TOPEXT,DBKP,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,AVX512F,AVX512DQ,RDSEED,SMAP,AVX512CD,SHA,AVX512BW,AVX512VL,XSAVEOPT
cpu0: 0KB 64b/line 1-way D-cache, 0KB 64b/line 1-way L2 cache, 0KB 64b/line
1-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 134MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD Ryzen 9 7950X3D 16-Core Processor, 4197.36 MHz, 19-61-02
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,LONG,LAHF,CMPLEG,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,SKINIT,TCE,TOPEXT,DBKP,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,AVX512F,AVX512DQ,RDSEED,SMAP,AVX512CD,SHA,AVX512BW,AVX512VL,XSAVEOPT
cpu1: 0KB 64b/line 1-way D-cache, 0KB 64b/line 1-way L2 cache, 0KB 64b/line
1-way L3 cache
cpu1: smt 0, core 0, package 1
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 32 pins
acpihpet0 at acpi0: 16777216 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xe0000000, bus 0-255
acpiprt0 at acpi0: bus 0 (PC00)
acpipci0 at acpi0 PC00
com0 at acpi0 COM1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at acpi0 COM2 addr 0x2f8/0x8 irq 3: ns16550a, 16 byte fifo
com2 at acpi0 COM3 addr 0x3e8/0x8 irq 4: ns16550a, 16 byte fifo
com3 at acpi0 COM4 addr 0x2e8/0x8 irq 3: ns16550a, 16 byte fifo
acpicmos0 at acpi0
"Bhyve_V_Gen_Counter_V1" at acpi0 not configured
pvbus0 at mainbus0: bhyve
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 unknown vendor 0x1275 product 0x1275 rev 0x00
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
virtio0 at pci0 dev 2 function 0 "Qumranet Virtio Network" rev 0x00
vio0 at virtio0: address 00:a0:98:6b:69:00
virtio0: msix per-VQ
virtio1 at pci0 dev 3 function 0 "Qumranet Virtio Storage" rev 0x00
vioblk0 at virtio1
scsibus1 at vioblk0: 1 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, >
sd0: 51200MB, 512 bytes/sector, 104857600 sectors
virtio1: msix per-VQ
vga1 at pci0 dev 29 function 0 unknown vendor 0xfb5d product 0x40fb rev 0x00
wsdisplay at vga1 not configured
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0 mux 1
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
efifb0 at mainbus0: 1024x768, 32bpp
wsdisplay0 at efifb0 mux 1
pckbd_enable: command error
wsdisplay0: screen 0-5 added (std, vt100 emulation)
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (52b8e1532127cb36.a) swap on sd0b dump on sd0b
usbdevs:
usbdevs: no USB controllers found
pcidump:
Domain /dev/pci0:
0:0:0: unknown unknown
0x0000: Vendor ID: 1275, Product ID: 1275
0x0004: Command: 0007, Status: 0010
0x0008: Class: 06 Bridge, Subclass: 00 Host,
Interface: 00, Revision: 00
0x000c: BIST: 00, Header Type: 00, Latency Timer: 00,
Cache Line Size: 00
0x0010: BAR empty (00000000)
0x0014: BAR empty (00000000)
0x0018: BAR empty (00000000)
0x001c: BAR empty (00000000)
0x0020: BAR empty (00000000)
0x0024: BAR empty (00000000)
0x0028: Cardbus CIS: 00000000
0x002c: Subsystem Vendor ID: 0000 Product ID: 0000
0x0030: Expansion ROM Base Address: 00000000
0x0038: 00000000
0x003c: Interrupt Pin: 00 Line: ff Min Gnt: 00 Max Lat: 00
0x0040: Capability 0x10: PCI Express
Max Payload Size: 128 / 128 bytes
Max Read Request Size: 128 bytes
Link Speed: 2.5 / 2.5 GT/s
Link Width: x1 / x1
0x0100: Enhanced Capability 0x00: Unknown
0x0000: 12751275 00100007 06000000 00000000
0x0010: 00000000 00000000 00000000 00000000
0x0020: 00000000 00000000 00000000 00000000
0x0030: 00000000 00000040 00000000 000000ff
0x0040: 00420010 00000000 00000000 00000411
0x0050: 00110000 00000000 00010000 00000000
0x0060: 00000000 00000000 00000000 00000000
0x0070: 00000000 00000000 00000000 00000000
0x0080: 00000000 00000000 00000000 00000000
0x0090: 00000000 00000000 00000000 00000000
0x00a0: 00000000 00000000 00000000 00000000
0x00b0: 00000000 00000000 00000000 00000000
0x00c0: 00000000 00000000 00000000 00000000
0x00d0: 00000000 00000000 00000000 00000000
0x00e0: 00000000 00000000 00000000 00000000
0x00f0: 00000000 00000000 00000000 00000000
0:1:0: Intel 82371SB ISA
0x0000: Vendor ID: 8086, Product ID: 7000
0x0004: Command: 0017, Status: 0000
0x0008: Class: 06 Bridge, Subclass: 01 ISA,
Interface: 00, Revision: 00
0x000c: BIST: 00, Header Type: 00, Latency Timer: 00,
Cache Line Size: 00
0x0010: BAR empty (00000000)
0x0014: BAR empty (00000000)
0x0018: BAR empty (00000000)
0x001c: BAR empty (00000000)
0x0020: BAR empty (00000000)
0x0024: BAR empty (00000000)
0x0028: Cardbus CIS: 00000000
0x002c: Subsystem Vendor ID: 0000 Product ID: 0000
0x0030: Expansion ROM Base Address: 00000000
0x0038: 00000000
0x003c: Interrupt Pin: 00 Line: ff Min Gnt: 00 Max Lat: 00
0x0000: 70008086 00000017 06010000 00000000
0x0010: 00000000 00000000 00000000 00000000
0x0020: 00000000 00000000 00000000 00000000
0x0030: 00000000 00000000 00000000 000000ff
0x0040: 00000000 00000000 00000000 00000000
0x0050: 00000000 00000000 00000000 00000000
0x0060: 0a0a0b0b 00000000 05808080 00000000
0x0070: 00000000 00000000 00000000 00000000
0x0080: 00000000 00000000 00000000 00000000
0x0090: 00000000 00000000 00000000 00000000
0x00a0: 00000000 00000000 00000000 00000000
0x00b0: 00000000 00000000 00000000 00000000
0x00c0: 00000000 00000000 00000000 00000000
0x00d0: 00000000 00000000 00000000 00000000
0x00e0: 00000000 00000000 00000000 00000000
0x00f0: 00000000 00000000 00000000 00000000
0:2:0: Qumranet Virtio Network
0x0000: Vendor ID: 1af4, Product ID: 1000
0x0004: Command: 0017, Status: 0010
0x0008: Class: 02 Network, Subclass: 00 Ethernet,
Interface: 00, Revision: 00
0x000c: BIST: 00, Header Type: 00, Latency Timer: 00,
Cache Line Size: 00
0x0010: BAR io addr: 0x00002080/0x0040
0x0014: BAR mem 32bit addr: 0xc1002000/0x00002000
0x0018: BAR empty (00000000)
0x001c: BAR empty (00000000)
0x0020: BAR empty (00000000)
0x0024: BAR empty (00000000)
0x0028: Cardbus CIS: 00000000
0x002c: Subsystem Vendor ID: 1af4 Product ID: 0001
0x0030: Expansion ROM Base Address: 00000000
0x0038: 00000000
0x003c: Interrupt Pin: 01 Line: 0a Min Gnt: 00 Max Lat: 00
0x0040: Capability 0x11: Extended Message Signalled Interrupts (MSI-X)
Enabled: yes; table size 3 (BAR 1:0)
0x004c: Capability 0x05: Message Signalled Interrupts (MSI)
Enabled: no; 1 vectors (1 enabled)
0x0000: 10001af4 00100017 02000000 00000000
0x0010: 00002081 c1002000 00000000 00000000
0x0020: 00000000 00000000 00000000 00011af4
0x0030: 00000000 00000040 00000000 0000010a
0x0040: 80024c11 00000001 00001001 00800005
0x0050: 00000000 00000000 00000000 00000000
0x0060: 00000000 00000000 00000000 00000000
0x0070: 00000000 00000000 00000000 00000000
0x0080: 00000000 00000000 00000000 00000000
0x0090: 00000000 00000000 00000000 00000000
0x00a0: 00000000 00000000 00000000 00000000
0x00b0: 00000000 00000000 00000000 00000000
0x00c0: 00000000 00000000 00000000 00000000
0x00d0: 00000000 00000000 00000000 00000000
0x00e0: 00000000 00000000 00000000 00000000
0x00f0: 00000000 00000000 00000000 00000000
0:3:0: Qumranet Virtio Storage
0x0000: Vendor ID: 1af4, Product ID: 1001
0x0004: Command: 0017, Status: 0010
0x0008: Class: 01 Mass Storage, Subclass: 00 SCSI,
Interface: 00, Revision: 00
0x000c: BIST: 00, Header Type: 00, Latency Timer: 00,
Cache Line Size: 00
0x0010: BAR io addr: 0x00002000/0x0080
0x0014: BAR mem 32bit addr: 0xc1000000/0x00002000
0x0018: BAR empty (00000000)
0x001c: BAR empty (00000000)
0x0020: BAR empty (00000000)
0x0024: BAR empty (00000000)
0x0028: Cardbus CIS: 00000000
0x002c: Subsystem Vendor ID: 1af4 Product ID: 0002
0x0030: Expansion ROM Base Address: 00000000
0x0038: 00000000
0x003c: Interrupt Pin: 01 Line: 0b Min Gnt: 00 Max Lat: 00
0x0040: Capability 0x11: Extended Message Signalled Interrupts (MSI-X)
Enabled: yes; table size 2 (BAR 1:0)
0x004c: Capability 0x05: Message Signalled Interrupts (MSI)
Enabled: no; 1 vectors (1 enabled)
0x0000: 10011af4 00100017 01000000 00000000
0x0010: 00002001 c1000000 00000000 00000000
0x0020: 00000000 00000000 00000000 00021af4
0x0030: 00000000 00000040 00000000 0000010b
0x0040: 80014c11 00000001 00001001 00800005
0x0050: 00000000 00000000 00000000 00000000
0x0060: 00000000 00000000 00000000 00000000
0x0070: 00000000 00000000 00000000 00000000
0x0080: 00000000 00000000 00000000 00000000
0x0090: 00000000 00000000 00000000 00000000
0x00a0: 00000000 00000000 00000000 00000000
0x00b0: 00000000 00000000 00000000 00000000
0x00c0: 00000000 00000000 00000000 00000000
0x00d0: 00000000 00000000 00000000 00000000
0x00e0: 00000000 00000000 00000000 00000000
0x00f0: 00000000 00000000 00000000 00000000
0:29:0: unknown unknown
0x0000: Vendor ID: fb5d, Product ID: 40fb
0x0004: Command: 0017, Status: 0010
0x0008: Class: 03 Display, Subclass: 00 VGA,
Interface: 00, Revision: 00
0x000c: BIST: 00, Header Type: 00, Latency Timer: 00,
Cache Line Size: 00
0x0010: BAR mem 32bit addr: 0xc1004000/0x00000080
0x0014: BAR mem 32bit addr: 0xc0000000/0x01000000
0x0018: BAR empty (00000000)
0x001c: BAR empty (00000000)
0x0020: BAR empty (00000000)
0x0024: BAR empty (00000000)
0x0028: Cardbus CIS: 00000000
0x002c: Subsystem Vendor ID: 0000 Product ID: 0000
0x0030: Expansion ROM Base Address: 00000000
0x0038: 00000000
0x003c: Interrupt Pin: 00 Line: ff Min Gnt: 00 Max Lat: 00
0x0040: Capability 0x05: Message Signalled Interrupts (MSI)
Enabled: no; 4 vectors (1 enabled)
0x0000: 40fbfb5d 00100017 03000000 00000000
0x0010: c1004000 c0000000 00000000 00000000
0x0020: 00000000 00000000 00000000 00000000
0x0030: 00000000 00000040 00000000 000000ff
0x0040: 00840005 00000000 00000000 00000000
0x0050: 00000000 00000000 00000000 00000000
0x0060: 00000000 00000000 00000000 00000000
0x0070: 00000000 00000000 00000000 00000000
0x0080: 00000000 00000000 00000000 00000000
0x0090: 00000000 00000000 00000000 00000000
0x00a0: 00000000 00000000 00000000 00000000
0x00b0: 00000000 00000000 00000000 00000000
0x00c0: 00000000 00000000 00000000 00000000
0x00d0: 00000000 00000000 00000000 00000000
0x00e0: 00000000 00000000 00000000 00000000
0x00f0: 00000000 00000000 00000000 00000000
acpidump:
begin-base64 644 APIC.3
QVBJQ2IAAAABkkJIWVZFIEJWQVBJQyAAAQAAAEJBU0wEBSIgAADg/gEAAAAACAAAAQAAAAAIAQEB
AAAAAQwAAAAAwP4AAAAAAgoAAAIAAAAFAAIKAAkJAAAADwAEBv8FAAE=
====
begin-base64 644 BGRT.7
QkdSVDgAAAABGElOVEVMIEVESzIgICAgAgAAACAgICATAAABAQABABjAzB4AAAAAnwEAAGMBAAA=
====
begin-base64 644 DSDT.2
RFNEVPkJAAACu0JIWVZFIEJWRFNEVCAgAQAAAElOVEwgECIgCF9TNV8SBQIKBQAIUElDTQAUDF9Q
SUMBcGhQSUNNEEaPX1NCX1uCTo5QQzAwCF9ISUQMQdAKAxQIX0JCTgCkAAhfQ1JTEUYJCpKIDQAC
DAAAAAAAAAAAAAEARwH4DPgMAQiIDQABDAMAAAAA9wwAAPgMiA0AAQwDAAAADf8fAAAAE4gNAAEM
AwAAACDfIAAA4ACHFwAADAEAAAAAAAAAwP//H8EAAAAAAAAgAYorAAAMAQAAAAAAAAAAAAAAAAgA
AAD///8fCAAAAAAAAAAAAAAAAAAAIAAAAAB5AAhQUFJUEigCEhIEDP//AgAALklTQV9MTktIABIS
BAz//wMAAC5JU0FfTE5LQQAIQVBSVBIaAhILBAz//wIAAAAKFxILBAz//wMAAAAKEBQYX1BSVACg
ClBJQ02kQVBSVKEGpFBQUlRbgkB9SVNBXwhfQURSDAAAAQBbgExQQ1ICAAsAAVuBM0xQQ1IAAEAw
UElSQQhQSVJCCFBJUkMIUElSRAgAIFBJUkUIUElSRghQSVJHCFBJUkgIW4ItS0JEXwhfSElEDEHQ
AwMIX0NSUxEYChVHAWAAYAABAUcBZABkAAEBIgIAeQBbgi1NT1VfCF9ISUQMQdAPEwhfQ1JTERgK
FUcBYABgAAEBRwFkAGQAAQEiABB5ABQuUElSVgGgCHtoCoAApAB7aAoPYKAHlWAKA6QAoAeTYAoI
pACgB5NgCg2kAKQBW4JKCkxOS0EIX0hJRAxB0AwPCF9VSUQBFBhfU1RBAKAMUElSVlBJUkGkCguh
BKQKCQhfUFJTEQkKBiP43hh5AAhDQjAxEQkKBiMAABh5AItDQjAxAUNJUkEUKV9DUlMAe1BJUkEK
j2CgDVBJUlZgeQFgQ0lSQaEHcABDSVJBpENCMDEUDV9ESVMAcAqAUElSQRQaX1NSUwGLaAFTSVJB
glNJUkFgcHZgUElSQVuCSwpMTktCCF9ISUQMQdAMDwhfVUlECgIUGF9TVEEAoAxQSVJWUElSQqQK
C6EEpAoJCF9QUlMRCQoGI/jeGHkACENCMDIRCQoGIwAAGHkAi0NCMDIBQ0lSQhQpX0NSUwB7UElS
QgqPYKANUElSVmB5AWBDSVJCoQdwAENJUkKkQ0IwMhQNX0RJUwBwCoBQSVJCFBpfU1JTAYtoAVNJ
UkKCU0lSQmBwdmBQSVJCW4JLCkxOS0MIX0hJRAxB0AwPCF9VSUQKAxQYX1NUQQCgDFBJUlZQSVJD
pAoLoQSkCgkIX1BSUxEJCgYj+N4YeQAIQ0IwMxEJCgYjAAAYeQCLQ0IwMwFDSVJDFClfQ1JTAHtQ
SVJDCo9goA1QSVJWYHkBYENJUkOhB3AAQ0lSQ6RDQjAzFA1fRElTAHAKgFBJUkMUGl9TUlMBi2gB
U0lSQ4JTSVJDYHB2YFBJUkNbgksKTE5LRAhfSElEDEHQDA8IX1VJRAoEFBhfU1RBAKAMUElSVlBJ
UkSkCguhBKQKCQhfUFJTEQkKBiP43hh5AAhDQjA0EQkKBiMAABh5AItDQjA0AUNJUkQUKV9DUlMA
e1BJUkQKj2CgDVBJUlZgeQFgQ0lSRKEHcABDSVJEpENCMDQUDV9ESVMAcAqAUElSRBQaX1NSUwGL
aAFTSVJEglNJUkRgcHZgUElSRFuCSwpMTktFCF9ISUQMQdAMDwhfVUlECgUUGF9TVEEAoAxQSVJW
UElSRaQKC6EEpAoJCF9QUlMRCQoGI/jeGHkACENCMDURCQoGIwAAGHkAi0NCMDUBQ0lSRRQpX0NS
UwB7UElSRQqPYKANUElSVmB5AWBDSVJFoQdwAENJUkWkQ0IwNRQNX0RJUwBwCoBQSVJFFBpfU1JT
AYtoAVNJUkWCU0lSRWBwdmBQSVJFW4JLCkxOS0YIX0hJRAxB0AwPCF9VSUQKBhQYX1NUQQCgDFBJ
UlZQSVJGpAoLoQSkCgkIX1BSUxEJCgYj+N4YeQAIQ0IwNhEJCgYjAAAYeQCLQ0IwNgFDSVJGFClf
Q1JTAHtQSVJGCo9goA1QSVJWYHkBYENJUkahB3AAQ0lSRqRDQjA2FA1fRElTAHAKgFBJUkYUGl9T
UlMBi2gBU0lSRoJTSVJGYHB2YFBJUkZbgksKTE5LRwhfSElEDEHQDA8IX1VJRAoHFBhfU1RBAKAM
UElSVlBJUkekCguhBKQKCQhfUFJTEQkKBiP43hh5AAhDQjA3EQkKBiMAABh5AItDQjA3AUNJUkcU
KV9DUlMAe1BJUkcKj2CgDVBJUlZgeQFgQ0lSR6EHcABDSVJHpENCMDcUDV9ESVMAcAqAUElSRxQa
X1NSUwGLaAFTSVJHglNJUkdgcHZgUElSR1uCSwpMTktICF9ISUQMQdAMDwhfVUlECggUGF9TVEEA
oAxQSVJWUElSSKQKC6EEpAoJCF9QUlMRCQoGI/jeGHkACENCMDgRCQoGIwAAGHkAi0NCMDgBQ0lS
SBQpX0NSUwB7UElSSAqPYKANUElSVmB5AWBDSVJIoQdwAENJUkikQ0IwOBQNX0RJUwBwCoBQSVJI
FBpfU1JTAYtoAVNJUkiCU0lSSGBwdmBQSVJIW4JIBVNJT18IX0hJRAxB0AwCCF9DUlMRQgQKPoYJ
AAEAAADgAAAAEEcB0ATQBAECRwFhAGEAAQFHAQAEAAQBCEcBsgCyAAEBRwGEAIQAAQFHAXIAcgAB
BnkAW4IrQ09NMQhfSElEDEHQBQEIX1VJRAEIX0NSUxEQCg1HAfgD+AMBCCIQAHkAW4IsQ09NMghf
SElEDEHQBQEIX1VJRAoCCF9DUlMREAoNRwH4AvgCAQgiCAB5AFuCLENPTTMIX0hJRAxB0AUBCF9V
SUQKAwhfQ1JTERAKDUcB6APoAwEIIhAAeQBbgixDT000CF9ISUQMQdAFAQhfVUlECgQIX0NSUxEQ
Cg1HAegC6AIBCCIIAHkAW4IlUlRDXwhfSElEDEHQCwAIX0NSUxEQCg1HAXAAcAABAiIAAXkAW4Ir
UElDXwhfSElEC0HQCF9DUlMRGAoVRwEgACAAAQJHAaAAoAABAiIEAHkAW4IlVElNUghfSElEDEHQ
AQAIX0NSUxEQCg1HAUAAQAABBCIBAHkAEDguX1NCX1BDMDBbgixIUEVUCF9ISUQMQdABAwhfVUlE
AAhfQ1JTEREKDoYJAAEAAND+AAQAAHkAEEwGX1NCX1uCRAZHRU5DCF9DSUQNVk1fR2VuX0NvdW50
ZXIAFB9fSElEAKQNQmh5dmVfVl9HZW5fQ291bnRlcl9WMQAIX1VJRAAIX0RETg1WTV9HZW5fQ291
bnRlcgAIQUREUhIIAgwAAAD/ABAZX0dQRRQTX0UwMACGXC5fU0JfR0VOQwqA
====
begin-base64 644 FACP.1
RkFDUBQBAAAFkkJIWVZFIEJWRkFDUCAAAQAAAEJBU0wEBSIgAOC/HwAwvx8AAAkAsgAAAKChAAAA
BAAAAAAAAAQEAAAAAAAAAAAAAAgEAAAMBAAAAAAAAAQCAAQCAAAAAAAAAAAAAAAAAAAAMhQAACUV
CAABCAAB+QwAAAAAAAAGAAABAAAAAAAAAAAAML8fAAAAAAEgAAIABAAAAAAAAAEAAAAAAAAAAAAA
AAEQAAIEBAAAAAAAAAEAAAAAAAAAAAAAAAEIAAAAAAAAAAAAAAEgAAMIBAAAAAAAAAEQAAEMBAAA
AAAAAAEAAAAAAAAAAAAAAAEIAAEAAAAAAAAAAAEIAAEAAAAAAAAAAAAAAAAAAAAA
====
begin-base64 644 HPET.4
SFBFVDgAAAABxEJIWVZFIEJWSFBFVCAAAQAAAEJBU0wEBSIgAQeGgAAAAAAAAND+AAAAAAAAAAE=
====
begin-base64 644 MCFG.5
TUNGRzwAAAAB5kJIWVZFIEJWTUNGRyAAAQAAAEJBU0wEBSIgAAAAAAAAAAAAAADgAAAAAAAAAP8A
AAAA
====
begin-base64 644 SPCR.6
U1BDUlAAAAABZUJIWVZFIEJWU1BDUiAAAQAAAEJBU0wEBSIgAAAAAAEIAAD4AwAAAAAAAAEEAAAA
AAcAAQMCAAAAAAAAAAAAAAAAAAAAAAA=
====
begin-base64 644 XSDT.0
WFNEVFQAAAAB3UJIWVZFIEJWRkFDUCAAAQAAACAgICATAAABAIC/HwAAAAAAcL8fAAAAAABgvx8A
AAAAAFC/HwAAAAAAQL8fAAAAAAAAMh4AAAAA
====
begin-base64 644 headers
ClJTRCBQVFI6IENoZWNrc3VtPTk1LCBPRU1JRD1CSFlWRSwgUmV2aXNpb249MiwgUnNkdEFkZHJl
c3M9MHgxZmJmOTA3NAoJTGVuZ3RoPTM2LCBYc2R0QWRkcmVzcz0weDAwMDAwMDAwMWZiZjkwZTgs
IEV4dGVuZGVkIENoZWNrc3VtPTEzNAoKClhTRFQ6IExlbmd0aD04NCwgUmV2aXNpb249MSwgQ2hl
Y2tzdW09MjIxLAoJT0VNSUQ9QkhZVkUsIE9FTSBUYWJsZSBJRD1CVkZBQ1AsIE9FTSBSZXZpc2lv
bj0weDEsCglDcmVhdG9yIElEPSwgQ3JlYXRvciBSZXZpc2lvbj0weDEwMDAwMTMKCgoJRW50cmll
cz17IDB4MDAwMDAwMDAxZmJmODAwMCwgMHgwMDAwMDAwMDFmYmY3MDAwLCAweDAwMDAwMDAwMWZi
ZjYwMDAsIDB4MDAwMDAwMDAxZmJmNTAwMCwgMHgwMDAwMDAwMDFmYmY0MDAwLCAweDAwMDAwMDAw
MWUzMjAwMDAgfQoKCglEU0RUPTB4MWZiZjMwMDAKCUlOVF9NT0RFTD1QSUMKCVNDSV9JTlQ9OQoJ
U01JX0NNRD0weGIyLCBBQ1BJX0VOQUJMRT0weGEwLCBBQ1BJX0RJU0FCTEU9MHhhMSwgUzRCSU9T
X1JFUT0weDAKCVBNMWFfRVZUX0JMSz0weDQwMC0weDQwMwoJUE0xYV9DTlRfQkxLPTB4NDA0LTB4
NDA1CglQTTJfVE1SX0JMSz0weDQwOC0weDQwYgoJUE0yX0dQRTBfQkxLPTB4NDBjLTB4NDBkCglQ
X0xWTDJfTEFUPTBtcywgUF9MVkwzX0xBVD0wbXMKCUZMVVNIX1NJWkU9MCwgRkxVU0hfU1RSSURF
PTAKCURVVFlfT0ZGU0VUPTAsIERVVFlfV0lEVEg9MAoJREFZX0FMUk09MCwgTU9OX0FMUk09MCwg
Q0VOVFVSWT01MAoJRmxhZ3M9e1dCSU5WRCxQUk9DX0MxLFNMUF9CVVRUT04sVE1SX1ZBTF9FWFR9
CgoKRFNEVDogTGVuZ3RoPTI1NTMsIFJldmlzaW9uPTIsIENoZWNrc3VtPTE4NywKCU9FTUlEPUJI
WVZFLCBPRU0gVGFibGUgSUQ9QlZEU0RULCBPRU0gUmV2aXNpb249MHgxLAoJQ3JlYXRvciBJRD1J
TlRMLCBDcmVhdG9yIFJldmlzaW9uPTB4MjAyMjEwMjAKCgpBUElDOiBMZW5ndGg9OTgsIFJldmlz
aW9uPTEsIENoZWNrc3VtPTE0NiwKCU9FTUlEPUJIWVZFLCBPRU0gVGFibGUgSUQ9QlZBUElDLCBP
RU0gUmV2aXNpb249MHgxLAoJQ3JlYXRvciBJRD1CQVNMLCBDcmVhdG9yIFJldmlzaW9uPTB4MjAy
MjA1MDQKCgpIUEVUOiBMZW5ndGg9NTYsIFJldmlzaW9uPTEsIENoZWNrc3VtPTE5NiwKCU9FTUlE
PUJIWVZFLCBPRU0gVGFibGUgSUQ9QlZIUEVULCBPRU0gUmV2aXNpb249MHgxLAoJQ3JlYXRvciBJ
RD1CQVNMLCBDcmVhdG9yIFJldmlzaW9uPTB4MjAyMjA1MDQKCgpNQ0ZHOiBMZW5ndGg9NjAsIFJl
dmlzaW9uPTEsIENoZWNrc3VtPTIzMCwKCU9FTUlEPUJIWVZFLCBPRU0gVGFibGUgSUQ9QlZNQ0ZH
LCBPRU0gUmV2aXNpb249MHgxLAoJQ3JlYXRvciBJRD1CQVNMLCBDcmVhdG9yIFJldmlzaW9uPTB4
MjAyMjA1MDQKCgpTUENSOiBMZW5ndGg9ODAsIFJldmlzaW9uPTEsIENoZWNrc3VtPTEwMSwKCU9F
TUlEPUJIWVZFLCBPRU0gVGFibGUgSUQ9QlZTUENSLCBPRU0gUmV2aXNpb249MHgxLAoJQ3JlYXRv
ciBJRD1CQVNMLCBDcmVhdG9yIFJldmlzaW9uPTB4MjAyMjA1MDQKCgpCR1JUOiBMZW5ndGg9NTYs
IFJldmlzaW9uPTEsIENoZWNrc3VtPTI0LAoJT0VNSUQ9SU5URUwsIE9FTSBUYWJsZSBJRD1FREsy
LCBPRU0gUmV2aXNpb249MHgyLAoJQ3JlYXRvciBJRD0sIENyZWF0b3IgUmV2aXNpb249MHgxMDAw
MDEzCgo=
====
