Hello,
after having backups running for few days, system crashes. I am using
rsnapshots for the backups.
The crash happened on snapshot from 22-03-2014 running i386.
Attached dmesg is slighlty newer, from 27-03-2014 (dmesg below).
An external backup hard drive is attached via USB.
Root's crontab looks like this:
* */8 * * * /usr/local/bin/rsnapshot hourly
>/dev/null 2>&1
0 3 * * * /usr/local/bin/rsnapshot daily
| \/usr/local/share/examples/rsnapshot/utils/rsnapreport.pl | mail -s
"Daily backup report" weezel@localhost
30 4 * * mon /usr/local/bin/rsnapshot weekly
| \/usr/local/share/examples/rsnapshot/utils/rsnapreport.pl | mail -s
"Weekly backup report" weezel@localhost
rsnapshots.conf:
config_version 1.2
snapshot_root /backup/.snapshots/
cmd_rm /bin/rm
cmd_rsync /usr/local/bin/rsync
cmd_ssh /usr/bin/ssh
cmd_logger /usr/bin/logger
cmd_du /usr/bin/du
cmd_rsnapshot_diff /usr/local/bin/rsnapshot-diff
cmd_preexec /sbin/mount -u -o rw,softdep /backup
cmd_postexec /sbin/mount -u -o ro /backup
interval hourly 2
interval daily 7
interval weekly 4
verbose 2
loglevel 3
lockfile /var/run/rsnapshot.pid
exclude_file /root/excludes.lst
link_dest 1
backup /home/ severi/
backup /etc/ severi/
backup /var/ severi/
Here are the debug messages I was able to retrieve via serial line. After
typing "show pool" it went to an unresponsive state.
ddb{1}> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
15482 1 15482 0 3 0x80 poll mpdscribble
28320 5251 28320 1000 3 0x83 poll python2.7
5251 679 5251 1000 3 0x8b pause ksh
8003 7745 8003 1000 3 0x83 select weechat
7745 679 7745 1000 3 0x8b pause ksh
679 1 679 1000 3 0x80 kqread tmux
16569 1 16569 0 3 0x83 ttyin getty
13445 1 13445 0 3 0x83 ttyin getty
20753 1 20753 0 3 0x83 ttyin getty
16989 1 16989 0 3 0x83 ttyin getty
3552 1 3552 0 3 0x83 ttyin getty
8692 1 8692 0 3 0x83 ttyin getty
27634 1 27634 0 3 0x80 select cron
1098 1 16421 649 3 0x90 netio nfcapd
25756 1 25756 0 3 0x80 kqread logfmon
32113 1 25756 0 3 0x4000080 nanosleep logfmon
15489 1 15489 0 3 0x90 select perl
3013 1 3013 560 3 0x90 poll mpd
5310 1 3013 560 3 0x4000090 poll mpd
20702 1 3013 560 3 0x4000090 thrsleep mpd
703 1 3013 560 3 0x4000090 thrsleep mpd
8100 1 3013 560 3 0x4000090 thrsleep mpd
12486 1 3013 560 3 0x4000090 thrsleep mpd
16813 1 22219 0 3 0x80 nanosleep smartd
12837 1 12837 67 3 0x82 poll fcgi-cgi
5563 1 11588 0 3 0x81 nanosleep perl
11326 1 11326 658 3 0x80 nanosleep vnstatd
21941 1 21941 99 3 0x90 poll sndiod
5152 1 5152 79 3 0x90 kqread tftpd
26890 24918 24918 67 3 0x90 kqread nginx
8363 24918 24918 67 3 0x90 kqread nginx
907 24918 24918 67 3 0x90 kqread nginx
22334 24918 24918 67 3 0x90 kqread nginx
24918 1 24918 0 3 0x88 pause nginx
29762 2597 2597 95 3 0x90 kqread smtpd
10648 2597 2597 95 3 0x90 kqread smtpd
22685 2597 2597 95 3 0x90 kqread smtpd
27851 2597 2597 95 3 0x90 kqread smtpd
31545 2597 2597 95 3 0x90 kqread smtpd
29168 2597 2597 95 3 0x90 kqread smtpd
23744 2597 2597 95 3 0x90 kqread smtpd
2198 2597 2597 103 3 0x90 kqread smtpd
2597 1 2597 0 3 0x80 kqread smtpd
6250 1 6250 77 3 0x90 poll dhcpd
30355 1 30355 0 3 0x80 select sshd
19848 22268 22268 0 3 0x80 nfsd nfsd
5320 22268 22268 0 3 0x80 nfsd nfsd
8860 22268 22268 0 3 0x80 nfsd nfsd
11333 22268 22268 0 3 0x80 nfsd nfsd
22268 1 22268 0 3 0x80 netcon nfsd
20177 1 20177 0 3 0x80 select mountd
19615 1 19615 28 3 0x90 poll portmap
20561 13562 20561 0 3 0x80 kqread npppd
13562 1 13562 82 3 0x90 kqread npppd
28044 31507 31507 68 3 0x90 select isakmpd
31507 1 31507 0 3 0x80 netio isakmpd
4563 1 4563 0 3 0x80 poll ntpd
16456 20728 16456 83 3 0x90 poll ntpd
20728 1 20728 83 3 0x90 poll ntpd
30132 27232 27232 53 3 0x90 kqread unbound
27232 1 27232 53 3 0x90 kqread unbound
27021 12821 12821 74 3 0x90 bpf pflogd
12821 1 12821 0 3 0x80 netio pflogd
2855 7831 7831 73 2 0x90 syslogd
7831 1 7831 0 3 0x80 netio syslogd
31640 1 31640 77 3 0x90 poll dhclient
12336 1 12336 0 3 0x80 poll dhclient
8537 0 0 0 3 0x4280 nfsidl nfsio
31801 0 0 0 3 0x4280 nfsidl nfsio
10485 0 0 0 3 0x4280 nfsidl nfsio
17987 0 0 0 3 0x4280 nfsidl nfsio
15730 0 0 0 3 0x4200 aiodoned aiodoned
*32348 0 0 0 7 0x4200 update
29589 0 0 0 3 0x4200 cleaner cleaner
1914 0 0 0 3 0x4200 reaper reaper
18929 0 0 0 3 0x4200 pgdaemon pagedaemon
7300 0 0 0 3 0x4200 bored crypto
7439 0 0 0 3 0x4200 pftm pfpurge
2333 0 0 0 3 0x4200 usbtsk usbtask
10095 0 0 0 3 0x4200 usbatsk usbatsk
9343 0 0 0 3 0x4200 bored sensors
17607 0 0 0 3 0x40004200 acpi0 acpi0
3474 0 0 0 7 0x40004200 idle3
19278 0 0 0 7 0x40004200 idle2
9013 0 0 0 3 0x40004200 idle1
10641 0 0 0 3 0x4200 bored systq
8631 0 0 0 3 0x4200 bored syswq
19377 0 0 0 7 0x40004200 idle0
22397 0 0 0 3 0x4200 kmalloc kmthread
1 0 1 0 3 0x82 wait init
0 -1 0 0 3 0x200 scheduler swapper
ddb{1}> trace
clear_inodedeps(d916da68,d810e400,f544cf3c,d040d68f,f544cf30) at clear_inodedep
s+0x11b
softdep_process_worklist(0,d9184000,3,d916da68,533437fe) at softdep_process_wor
klist+0x4ff
sched_sync(d916da68) at sched_sync+0xf0
Bad frame pointer: 0xd0d31e98
ddb{1}> machine ddbcpu 0
Stopped at Debugger+0x4: popl %ebp
ddb{0}> show pool
POOL
OpenBSD 5.5-current (GENERIC.MP) #9: Wed Mar 19 18:15:43 MDT 2014
[email protected]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Atom(TM) CPU D2700 @ 2.13GHz ("GenuineIntel" 686-class) 2.13 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,LONG,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,LAHF,PERF,ITSC
real mem = 2135228416 (2036MB)
avail mem = 2087911424 (1991MB)
User Kernel Config
UKC> ddis\^H \^H\^H \^H\^H \^Hisable acpiec
494 acpiec* disabled
UKC> quit
Continuing...
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 10/20/10, SMBIOS rev. 2.7 @
0xe9670 (51 entries)
bios0: vendor American Megatrends Inc. version "4.6.4" date 04/09/2013
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC MCFG SLIC HPET SSDT
acpi0: wakeup devices P0P8(S4) PS2K(S1) PS2M(S1) USB0(S4) USB1(S4)
USB2(S4) USB3(S4) USB7(S4) PXSX(S4) RP01(S4) PXSX(S4) RP03(S4)
PXSX(S4) RP04(S4) PXSX(S4) RP02(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
mtrr: Pentium Pro MTRR support, 7 var ranges, 88 fixed ranges
cpu0: apic clock running at 133MHz
cpu0: mwait min=64, max=64, C-substates=0.1.0.0.0, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Atom(TM) CPU D2700 @ 2.13GHz ("GenuineIntel" 686-class) 2.13 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,LONG,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,LAHF,PERF,ITSC
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Atom(TM) CPU D2700 @ 2.13GHz ("GenuineIntel" 686-class) 2.13 GHz
cpu2:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,LONG,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,LAHF,PERF,ITSC
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Atom(TM) CPU D2700 @ 2.13GHz ("GenuineIntel" 686-class) 2.13 GHz
cpu3:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,LONG,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,LAHF,PERF,ITSC
ioapic0 at mainbus0: apid 4 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 5 (P0P8)
acpiprt2 at acpi0: bus 1 (RP01)
acpiprt3 at acpi0: bus 3 (RP03)
acpiprt4 at acpi0: bus 4 (RP04)
acpiprt5 at acpi0: bus 2 (RP02)
acpiec at acpi0 not configured
acpicpu0 at acpi0: C1
acpicpu1 at acpi0: C1
acpicpu2 at acpi0: C1
acpicpu3 at acpi0: C1
acpitz0 at acpi0: critical temperature is 70 degC
acpipwrres0 at acpi0: FN00, resource for FAN0
acpitz1 at acpi0: critical temperature is 127 degC
acpitz2 at acpi0: critical temperature is 100 degC
acpibat0 at acpi0: BAT0 model "CRB Battery 0" serial Battery 0 type
Fake oem "-Virtual Battery 0-"
acpibat1 at acpi0: BAT1 not present
acpibtn0 at acpi0: LID0
acpibtn1 at acpi0: PWRB
acpibtn2 at acpi0: SLPB
acpivideo0 at acpi0: GFX0
bios0: ROM list: 0xc0000/0xf400!
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 vendor "Intel", unknown product 0x0bf3 rev 0x03
vga1 at pci0 dev 2 function 0 vendor "Intel", unknown product 0x0be2 rev 0x09
intagp at vga1 not configured
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
azalia0 at pci0 dev 27 function 0 "Intel 82801GB HD Audio" rev 0x02: msi
azalia0: codecs: Realtek ALC662, Intel/0x2880, using Realtek ALC662
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x02: apic 4 int 16
pci1 at ppb0 bus 1
ppb1 at pci0 dev 28 function 1 "Intel 82801GB PCIE" rev 0x02: apic 4 int 17
pci2 at ppb1 bus 2
re0 at pci2 dev 0 function 0 "Realtek 8168" rev 0x06:
RTL8168E/8111E-VL (0x2c80), msi, address 00:30:18:a4:f8:e2
rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 5
ppb2 at pci0 dev 28 function 2 "Intel 82801GB PCIE" rev 0x02: apic 4 int 18
pci3 at ppb2 bus 3
re1 at pci3 dev 0 function 0 "Realtek 8168" rev 0x06:
RTL8168E/8111E-VL (0x2c80), msi, address 00:30:18:a4:f8:e3
rgephy1 at re1 phy 7: RTL8169S/8110S PHY, rev. 5
ppb3 at pci0 dev 28 function 3 "Intel 82801GB PCIE" rev 0x02: apic 4 int 19
pci4 at ppb3 bus 4
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x02: apic 4 int 23
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x02: apic 4 int 19
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x02: apic 4 int 18
uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x02: apic 4 int 16
ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x02: apic 4 int 23
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb4 at pci0 dev 30 function 0 "Intel 82801BAM Hub-to-PCI" rev 0xe2
pci5 at ppb4 bus 5
em0 at pci5 dev 1 function 0 "Intel 82541GI" rev 0x05: apic 4 int 17,
address 00:1b:21:14:3e:dd
pcib0 at pci0 dev 31 function 0 "Intel NM10 LPC" rev 0x02
ahci0 at pci0 dev 31 function 2 "Intel 82801GR AHCI" rev 0x02: msi, AHCI 1.1
scsibus0 at ahci0: 32 targets
sd0 at scsibus0 targ 0 lun 0: <ATA, WDC WD15EARX-00P, 51.0> SCSI3
0/direct fixed naa.50014ee003098d0d
sd0: 1430799MB, 512 bytes/sector, 2930277168 sectors
ichiic0 at pci0 dev 31 function 3 "Intel 82801GB SMBus" rev 0x02: apic 4 int 19
iic0 at ichiic0
spdmem0 at iic0 addr 0x51: 2GB DDR3 SDRAM PC3-10600 SO-DIMM
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci3: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
umass0 at uhub0 port 5 configuration 1 interface 0 "JMicron USB to
ATA/ATAPI bridge" rev 2.00/1.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus1 at umass0: 2 targets, initiator 0
sd1 at scsibus1 targ 1 lun 0: <ST350063, 0A, > SCSI2 0/direct fixed
serial.152d233800630A222222
sd1: 476940MB, 512 bytes/sector, 976773168 sectors
uhidev0 at uhub4 port 2 configuration 1 interface 0 "American Power
Conversion Back-UPS ES/CyberFort 350 FW: 3.1.I USB FW: d1" rev
1.10/1.00 addr 2
uhidev0: iclass 3/0, 64 report ids
uhid0 at uhidev0 reportid 1: input=0, output=0, feature=1
uhid1 at uhidev0 reportid 2: input=0, output=0, feature=1
uhid2 at uhidev0 reportid 3: input=0, output=0, feature=1
uhid3 at uhidev0 reportid 4: input=0, output=0, feature=1
uhid4 at uhidev0 reportid 5: input=0, output=0, feature=1
uhid5 at uhidev0 reportid 6: input=3, output=0, feature=3
uhid6 at uhidev0 reportid 7: input=0, output=0, feature=2
uhid7 at uhidev0 reportid 8: input=0, output=0, feature=2
uhid8 at uhidev0 reportid 9: input=0, output=0, feature=2
uhid9 at uhidev0 reportid 10: input=0, output=0, feature=1
uhid10 at uhidev0 reportid 11: input=0, output=0, feature=1
uhid11 at uhidev0 reportid 12: input=5, output=0, feature=5
uhid12 at uhidev0 reportid 13: input=0, output=0, feature=3
uhid13 at uhidev0 reportid 14: input=0, output=0, feature=3
uhid14 at uhidev0 reportid 15: input=0, output=0, feature=3
uhid15 at uhidev0 reportid 16: input=0, output=0, feature=3
uhid16 at uhidev0 reportid 17: input=0, output=0, feature=3
uhid17 at uhidev0 reportid 18: input=0, output=0, feature=3
uhid18 at uhidev0 reportid 19: input=1, output=0, feature=1
uhid19 at uhidev0 reportid 20: input=2, output=0, feature=2
uhid20 at uhidev0 reportid 28: input=0, output=0, feature=3
uhid21 at uhidev0 reportid 53: input=0, output=0, feature=1
uhid22 at uhidev0 reportid 64: input=0, output=0, feature=1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (082c572cf96776d9.a) swap on sd0b dump on sd0b
Please CC me since I'm not subscribed to bugs, thanks.
--
Regards,
Ville Valkonen
