Hi list,
I'm currently evaluating unionfs for a cluster setup with a single NFS
root image:
I create a tmpfs on /local and overlay /etc with /local/etc and /root
with /local/root.
/var is overlayed by /local/var, which is a ext3 or xfs harddisk partition.
The rest remains read only.
The system is Debian Sarge, the Kernel is a "backported"
2.6.12-1-686-smp with the default Debian config.
I've seen some issues with unionfs 1.0.11 - lockups of postfix, oops on
shutdown (umount of /var), so I tried 1.0.14 (Debian unionfs-source from
unstable)
Unfortunately the problems actually got worse. On some boots unionfs
oopses directly on first invocation (/etc) and locks up the system,
while other boots succeed.
Additionally on a box with 156 MB oopses occured after the machine was
up for a while - I assume these may be memory related (swapping vs.
tmpfs?). These dissappeared after upgrading the machine to 512 MB.
Surprisingly I do not see errors on diskless boxes where /var is on
tmpfs as well (even though most errors occur before creation of the /var
unionfs ...)
Thankful for any clues,
Christopher
- -
These are the errors I compiled in the last two days:
# On boot:
unionfs: no version for "struct_module" found: kernel tainted.
Registering unionfs 1.0.14
...
ASSERTION FAILED (atomic_read(&d->d_count)) > 0 at
/usr/src/modules/unionfs/unionfs_debugmacros.h:416 (__dtohd_index)
called by /usr/src/modules/unionfs/dentry.c:171 (unionfs_d_revalidate)
Unable to handle kernel NULL pointer dereference at virtual address 00000000
printing eip:
dd430aa2
*pde = 00000000
Oops: 0002 [#1]
SMP
Modules linked in: agpgart unionfs jfs xfs exportfs ext2 ext3 jbd
mbcache r8169 e1000 forcedeth yellowfin via_rhine tulip de4x5 sis900
ne2k_pci hp100 epic100 eepro100 e100 dmfe 8139too 8139cp 3c59x mii 3c509
8390 af_packet nfs lockd sunrpc sd_mod 3w_9xxx sg 3w_xxxx aic7xxx
scsi_transport_spi scsi_mod unix fbcon tileblit font bitblit vesafb
cfbcopyarea cfbimgblt cfbfillrect softcursor capability commoncap
ide_disk ide_generic atiixp piix ide_core
CPU: 0
EIP: 0060:[<dd430aa2>] Tainted: GF VLI
EFLAGS: 00010296 (2.6.12-1-686-smp)
EIP is at unionfs_d_revalidate+0xfe42/0x12c20 [unionfs]
eax: 000000c3 ebx: dac1d600 ecx: 00000000 edx: 00000286
esi: dd59ab40 edi: 00000001 ebp: 00000001 esp: da541d88
ds: 007b es: 007b ss: 0068
Process udev (pid: 1717, threadinfo=da540000 task=da47aa40)
Stack: dd59ab00 dd59adc0 dd59ab40 000001a0 dd597af8 dd59aa80 000000ab
dd597a9f
0000002a db28888c 00000001 da541f58 00000000 00000000 00000001
00000000
00000001 00000001 da94b62c dac7a884 000000ce dd433a40 dac7a884
da541f58
Call Trace:
[<dd433a40>] unionfs_d_revalidate_wrap+0x1c0/0xb10 [unionfs]
[<c016ce3f>] do_lookup+0x6f/0xb0
[<c016d5c7>] __link_path_walk+0x747/0xe60
[<c016dd29>] link_path_walk+0x49/0xe0
[<c01cb8f1>] strncpy_from_user+0x41/0x70
[<c015d832>] get_unused_fd+0xa2/0xd0
[<c016e084>] path_lookup+0x94/0x170
[<c011595c>] do_page_fault+0x19c/0x5d5
[<c016e8c6>] open_namei+0x86/0x670
[<c01157c0>] do_page_fault+0x0/0x5d5
[<c015d55e>] filp_open+0x3e/0x70
[<c01cb8f1>] strncpy_from_user+0x41/0x70
[<c015d832>] get_unused_fd+0xa2/0xd0
[<c015d959>] sys_open+0x49/0x90
[<c0103289>] syscall_call+0x7/0xb
Code: 44 24 18 b8 80 aa 59 dd 89 44 24 14 b8 f8 7a 59 dd 89 44 24 10 b8
a0 01 00 00 89 44 24 0c 89 74 24 08 89 4c 24 04 e8fe cf ce e2 <c6> 05 00
00 00 00 00 e9 a6 63 ff ff c7 04 24 00 ac 59 dd b8 9f
# On boot, different box:
ASSERTION FAILED (atomic_read(&hidden_inode->i_count)) > 0 at
/usr/src/modules/unionfs/unionfs_debugmacros.h:92 (__itohi_index) called
by /usr/src/modules/unionfs/print.c:412 (fist_checkinode)
Unable to handle kernel NULL pointer dereference at virtual address 00000000
printing eip:
d13981fa
*pde = 00000000
Oops: 0002 [#1]
SMP
Modules linked in: tsdev mousedev psmouse evdev floppy pcspkr rtc
pdc202xx_old via686a i2c_sensor i2c_viapro i2c_core pci_hotplug via_agp
sr_mod cdrom sym53c8xx snd_via82xx gameport snd_ac97_codec snd_pcm_oss
snd_mixer_oss snd_pcm snd_timer snd_page_alloc snd_mpu401_uart
snd_rawmidi snd_seq_device snd soundcore uhci_hcd usbcore via82cxxx
parport_pc parport agpgart unionfs jfs xfs exportfs ext2 ext3 jbd
mbcache r8169 e1000 forcedeth yellowfin via_rhine tulip de4x5 sis900
ne2k_pci hp100 epic100 eepro100 e100 dmfe 8139too 8139cp 3c59x mii 3c509
8390 af_packet nfs lockd sunrpc sd_mod 3w_9xxx sg 3w_xxxx aic7xxx
scsi_transport_spi scsi_mod unix fbcon tileblit font bitblit vesafb
cfbcopyarea cfbimgblt cfbfillrect softcursor capabilitycommoncap
ide_disk ide_generic atiixp piix ide_core
CPU: 0
EIP: 0060:[<d13981fa>] Tainted: GFS VLI
EFLAGS: 00010296 (2.6.12-1-686-smp)
EIP is at fist_checkinode+0x2da/0x350 [unionfs]
eax: 000000c7 ebx: ce145ce0 ecx: 00000000 edx: 00000296
esi: ce145cac edi: d13a0d20 ebp: d13a1f66 esp: cfd23e04
ds: 007b es: 007b ss: 0068
Process kswapd0 (pid: 101, threadinfo=cfd22000 task=cff19590)
Stack: d13a0ca0 d13a0d20 d13a0ce0 0000005c d139b2a1 d13a0b80 0000019c
d139b36e
d13916d4 ce94e638 ce94e62c ce145ce0 d12a2ea0 ce510170 00000035
d12a2f10
ce145ce0 d13a1f66 00000072 00000004 d13a1f05 d13a96a0 d13994e5
d139d200
Call Trace:
[<d13916d4>] fist_dprint_internal+0x14/0x80 [unionfs]
[<d12a2ea0>] unionfs_delete_inode+0x0/0xf0 [unionfs]
[<d12a2f10>] unionfs_delete_inode+0x70/0xf0 [unionfs]
[<c017a325>] generic_delete_inode+0xb5/0x160
[<c01cb5b1>] _atomic_dec_and_lock+0x31/0x50
[<c017a5a3>] iput+0x63/0x90
[<d123abe3>] unionfs_d_iput+0x63/0xbb [unionfs]
[<c0177811>] prune_dcache+0x151/0x1a0
[<c0177c4f>] shrink_dcache_memory+0x1f/0x50
[<c0149e45>] shrink_slab+0x155/0x190
[<c014b28f>] balance_pgdat+0x2ff/0x3d0
[<c0133120>] prepare_to_wait+0x20/0x70
[<c014b449>] kswapd+0xe9/0x120
[<c0133250>] autoremove_wake_function+0x0/0x60
[<c0133250>] autoremove_wake_function+0x0/0x60
[<c014b360>] kswapd+0x0/0x120
[<c0101485>] kernel_thread_helper+0x5/0x10
Code: 0b 3a d1 89 44 24 14 b8 a1 b2 39 d1 89 44 24 10 b8 5c 00 00 00 89
44 24 0c b8 e0 0c 3a d1 89 44 24 08 89 7c 24 04 e8a6 58 d8 ee <c6> 05 00
00 00 00 00 e9 69 fd ff ff 89 6c 24 18 be a0 13 3a d1
# On boot, third box:
ASSERTION FAILED (atomic_read(&hidden_file->f_count)) > 0 at
/usr/src/modules/unionfs/unionfs_debugmacros.h:39 (__ftohf_index) called
by /usr/src/modules/unionfs/print.c:256 (fist_print_file)
Unable to handle kernel NULL pointer dereference at virtual address 00000000
printing eip:
f982c408
*pde = 00000000
Oops: 0002 [#1]
SMP
Modules linked in: unionfs jfs xfs exportfs ext2 ext3 jbd mbcache r8169
e1000 forcedeth yellowfin via_rhine tulip de4x5 sis900 ne2k_pci hp100
epic100 eepro100 e100 dmfe 8139too 8139cp 3c59x mii 3c509 8390 af_packet
nfs lockd sunrpc sd_mod 3w_9xxx sg 3w_xxxx aic7xxx scsi_transport_spi
scsi_mod unix fbcon tileblit font bitblit vesafb cfbcopyarea cfbimgblt
cfbfillrect softcursor capability commoncap ide_disk ide_generic atiixp
piix ide_core
CPU: 1
EIP: 0060:[<f982c408>] Tainted: GF VLI
EFLAGS: 00010292 (2.6.12-1-686-smp)
EIP is at fist_print_file+0x188/0x2e0 [unionfs]
eax: 000000c6 ebx: 00000001 ecx: 00000000 edx: 00000292
esi: dfd15900 edi: f7a31e94 ebp: f78c9c80 esp: f7a31e74
ds: 007b es: 007b ss: 0068
Process rsync (pid: 1621, threadinfo=f7a30000 task=f79a8a40)
Stack: f983aca0 f983ade0 f983ace0 00000027 f98352ed f983ab80 00000100
f98352dd
00000000 00000031 00000001 00000000 00000001 ffffffff dfd15980
f7ec8210
00000000 f9831920 00000000 00000171 f7eda400 f79a8a40 00000000
00000000
Call Trace:
[<f9831920>] fist_print_dentry+0x20/0x30 [unionfs]
[<c0133250>] autoremove_wake_function+0x0/0x60
[<f96d52bf>] unionfs_read+0x11f/0x410 [unionfs]
[<c015e395>] vfs_read+0xe5/0x160
[<c015e6b1>] sys_read+0x51/0x80
[<c0103289>] syscall_call+0x7/0xb
Code: 52 83 f9 89 44 24 10 b8 27 00 00 00 89 44 24 0c b8 e0 ac 83 f9 89
44 24 08 b8 e0 ad 83 f9 89 54 24 1c 89 44 24 04 e898 16 8f c6 <c6> 05 00
00 00 00 00 e9 6a ff ff ff 89 5c 24 18 8b 84 24 b4 00
<2>ASSERTION FAILED (atomic_read(&hidden_file->f_count)) > 0 at
/usr/src/modules/unionfs/unionfs_debugmacros.h:39 (__ftohf_index) called
by /usr/src/modules/unionfs/commonfops.c:706 (unionfs_flush)
Unable to handle kernel NULL pointer dereference at virtual address 00000000
printing eip:
f982a437
*pde = 00000000
Oops: 0002 [#2]
SMP
Modules linked in: unionfs jfs xfs exportfs ext2 ext3 jbd mbcache r8169
e1000 forcedeth yellowfin via_rhine tulip de4x5 sis900 ne2k_pci hp100
epic100 eepro100 e100 dmfe 8139too 8139cp 3c59x mii 3c509 8390 af_packet
nfs lockd sunrpc sd_mod 3w_9xxx sg 3w_xxxx aic7xxx scsi_transport_spi
scsi_mod unix fbcon tileblit font bitblit vesafb cfbcopyarea cfbimgblt
cfbfillrect softcursor capability commoncap ide_disk ide_generic atiixp
piix ide_core
CPU: 1
EIP: 0060:[<f982a437>] Tainted: GF VLI
EFLAGS: 00010286 (2.6.12-1-686-smp)
EIP is at unionfs_flush+0x3d27/0x40f0 [unionfs]
eax: 000000c6 ebx: dfd15900 ecx: 00000000 edx: 00000282
esi: f78cadac edi: f983a460 ebp: 00000001 esp: f7a31c8c
ds: 007b es: 007b ss: 0068
Process rsync (pid: 1621, threadinfo=f7a30000 task=f79a8a40)
Stack: f983a0c0 f983a460 f983a100 00000027 f9834de2 f9839fc0 000002c2
f9834ebb
000002b5 f7ec8080 00000001 00000000 f78c9c80 00000000 f79ffd00
00000001
c015da46 f78c9c80 f79ffd00 00000007 00000003 f79ffd00 c011f6d4
f78c9c80
Call Trace:
[<c015da46>] filp_close+0x76/0x90
[<c011f6d4>] put_files_struct+0x64/0xd0
[<c01203f6>] do_exit+0xf6/0x370
[<c01045fc>] die+0x17c/0x180
[<c0115a9a>] do_page_fault+0x2da/0x5d5
[<c0103d4c>] apic_timer_interrupt+0x1c/0x24
[<c011dde3>] release_console_sem+0x53/0xc0
[<c011de0d>] release_console_sem+0x7d/0xc0
[<c011dc5d>] vprintk+0x19d/0x250
[<f982c1f8>] fist_print_generic_file3+0x308/0x360 [unionfs]
[<c01157c0>] do_page_fault+0x0/0x5d5
[<c0103e17>] error_code+0x4f/0x54
[<f983007b>] __fist_print_dentry+0x3acb/0x5350 [unionfs]
[<f982c408>] fist_print_file+0x188/0x2e0 [unionfs]
[<f9831920>] fist_print_dentry+0x20/0x30 [unionfs]
[<c0133250>] autoremove_wake_function+0x0/0x60
[<f96d52bf>] unionfs_read+0x11f/0x410 [unionfs]
[<c015e395>] vfs_read+0xe5/0x160
[<c015e6b1>] sys_read+0x51/0x80
[<c0103289>] syscall_call+0x7/0xb
Code: 9f 83 f9 89 44 24 14 b8 e2 4d 83 f9 89 44 24 10 b8 27 00 00 00 89
44 24 0c b8 00 a1 83 f9 89 44 24 08 89 7c 24 04 e869 36 8f c6 <c6> 05 00
00 00 00 00 e9 fb c4 ff ff 8b 5c a8 10 e9 e3 c4 ff ff
# Errors later on (on the 256 MB machine):
unionfs: no version for "struct_module" found: kernel tainted.
Registering unionfs 1.0.14
...
ASSERTION FAILED (atomic_read(&d->d_count)) > 0 at
/usr/src/modules/unionfs/unionfs_debugmacros.h:416 (__dtohd_index)
called by /usr/src/modules/unionfs/print.c:319 (__fist_print_dentry)
Unable to handle kernel NULL pointer dereference at virtual address 00000000
printing eip:
d1393ab0
*pde = 00000000
Oops: 0002 [#1]
SMP
Modules linked in: autofs4 thermal fan button processor ac battery ipv6
tsdev mousedev psmouse evdev floppy pcspkr rtc pdc202xx_old via686a
i2c_sensor i2c_viapro i2c_core pci_hotplug via_agp sr_mod cdrom
sym53c8xx snd_via82xx gameport snd_ac97_codec snd_pcm_oss snd_mixer_oss
snd_pcm snd_timer snd_page_alloc snd_mpu401_uart snd_rawmidi
snd_seq_device snd soundcore uhci_hcd usbcore via82cxxx parport_pc
parport agpgart unionfs jfs xfs exportfs ext2 ext3 jbd mbcache r8169
e1000 forcedeth yellowfin via_rhine tulip de4x5 sis900 ne2k_pci hp100
epic100 eepro100 e100 dmfe 8139too 8139cp 3c59x mii 3c509 8390 af_packet
nfs lockd sunrpc sd_mod 3w_9xxx sg 3w_xxxx aic7xxx scsi_transport_spi
scsi_mod unix fbcon tileblit font bitblit vesafb cfbcopyarea cfbimgblt
cfbfillrect softcursor capability commoncap ide_disk ide_generic atiixp
piix ide_core
CPU: 0
EIP: 0060:[<d1393ab0>] Tainted: GFS VLI
EFLAGS: 00210282 (2.6.12-1-686-smp)
EIP is at __fist_print_dentry+0x1500/0x5350 [unionfs]
eax: 000000c1 ebx: d139b2fb ecx: 00000000 edx: 00200286
esi: cf758200 edi: 00000001 ebp: ceb11818 esp: c6dffc0c
ds: 007b es: 007b ss: 0068
Process konsole (pid: 5977, threadinfo=c6dfe000 task=c702d0a0)
Stack: d13a0ca0 d13a0f20 d13a0ce0 000001a0 d139b344 d13a0b80 0000013f
d139b2fb
ffffffff ffffffff ce998720 c017a580 00000152 00000152 ce998720
d12a3805
d139d200 d139953b 00000152 00000005 d13a1f19 d13a96a0 d139953b
d139d200
Call Trace:
[<c017a580>] iput+0x40/0x90
[<d12a3805>] unionfs_clear_inode+0x165/0x230 [unionfs]
[<d13916d4>] fist_dprint_internal+0x14/0x80 [unionfs]
[<d13916d4>] fist_dprint_internal+0x14/0x80 [unionfs]
[<c0179426>] clear_inode+0xe6/0x130
[<d12a3b2b>] unionfs_destroy_inode+0xcb/0xe0 [unionfs]
[<d13916d4>] fist_dprint_internal+0x14/0x80 [unionfs]
[<d1235556>] unionfs_d_release+0x1c6/0x57f0 [unionfs]
[<c0177216>] d_free+0x36/0x40
[<c01777e5>] prune_dcache+0x125/0x1a0
[<c0177c4f>] shrink_dcache_memory+0x1f/0x50
[<c0149e45>] shrink_slab+0x155/0x190
[<c014aeb9>] try_to_free_pages+0xe9/0x1c0
[<c0143c08>] __alloc_pages+0x1a8/0x430
[<c0159a53>] shmem_getpage+0x153/0x850
[<c015a212>] shmem_nopage+0xc2/0xf0
[<c014f08b>] do_no_page+0xbb/0x330
[<c014f506>] handle_mm_fault+0xf6/0x170
[<c011595c>] do_page_fault+0x19c/0x5d5
[<c01270f1>] update_process_times+0x91/0x120
[<c01272c6>] run_timer_softirq+0x126/0x1c0
[<c013d560>] handle_IRQ_event+0x30/0x70
[<c0122cf6>] __do_softirq+0xd6/0xf0
[<c01157c0>] do_page_fault+0x0/0x5d5
[<c0103e17>] error_code+0x4f/0x54
Code: 5c 24 1c b8 a0 01 00 00 89 44 24 0c b8 e0 0c 3a d1 89 44 24 08 b8
20 0f 3a d1 89 4c 24 18 89 54 24 14 89 44 24 04 e8f0 9f d8 ee <c6> 05 00
00 00 00 00 e9 73 ff ff ff c7 04 24 40 0f 3a d1 b8 80
ASSERTION FAILED (atomic_read(&hidden_inode->i_count)) > 0 at
/usr/src/modules/unionfs/unionfs_debugmacros.h:92 (__itohi_index) called
by /usr/src/modules/unionfs/print.c:412 (fist_checkinode)
Unable to handle kernel NULL pointer dereference at virtual address 00000000
printing eip:
d13981fa
*pde = 0bb2c067
*pte = 00000000
Oops: 0002 [#1]
SMP
Modules linked in: autofs4 thermal fan button processor ac battery ipv6
tsdev mousedev psmouse evdev floppy pcspkr rtc pdc202xx_old via686a
i2c_sensor i2c_viapro i2c_core pci_hotplug via_agp sr_mod cdrom
sym53c8xx snd_via82xx gameport snd_ac97_codec snd_pcm_oss snd_mixer_oss
snd_pcm snd_timer snd_page_alloc snd_mpu401_uart snd_rawmidi
snd_seq_device snd soundcore uhci_hcd usbcore via82cxxx parport_pc
parport agpgart unionfs jfs xfs exportfs ext2 ext3 jbd mbcache r8169
e1000 forcedeth yellowfin via_rhine tulip de4x5 sis900 ne2k_pci hp100
epic100 eepro100 e100 dmfe 8139too 8139cp 3c59x mii 3c509 8390 af_packet
nfs lockd sunrpc sd_mod 3w_9xxx sg 3w_xxxx aic7xxx scsi_transport_spi
scsi_mod unix fbcon tileblit font bitblit vesafb cfbcopyarea cfbimgblt
cfbfillrect softcursor capability commoncap ide_disk ide_generic atiixp
piix ide_core
CPU: 0
EIP: 0060:[<d13981fa>] Tainted: GFS VLI
EFLAGS: 00010296 (2.6.12-1-686-smp)
EIP is at fist_checkinode+0x2da/0x350 [unionfs]
eax: 000000c7 ebx: cea89ce0 ecx: ce963a00 edx: 00000296
esi: cea89cac edi: d13a0d20 ebp: d13a1f66 esp: cfd23e04
ds: 007b es: 007b ss: 0068
Process kswapd0 (pid: 101, threadinfo=cfd22000 task=cff19590)
Stack: d13a0ca0 d13a0d20 d13a0ce0 0000005c d139b2a1 d13a0b80 0000019c
d139b36e
d13916d4 ce5fc4b8 ce5fc4ac cea89ce0 d12a2ea0 ce761170 0000007c
d12a2f10
cea89ce0 d13a1f66 00000072 00000004 d13a1f05 d13a96a0 d13994e5
d139d200
Call Trace:
[<d13916d4>] fist_dprint_internal+0x14/0x80 [unionfs]
[<d12a2ea0>] unionfs_delete_inode+0x0/0xf0 [unionfs]
[<d12a2f10>] unionfs_delete_inode+0x70/0xf0 [unionfs]
[<c017a325>] generic_delete_inode+0xb5/0x160
[<c01cb5b1>] _atomic_dec_and_lock+0x31/0x50
[<c017a5a3>] iput+0x63/0x90
[<d123abe3>] unionfs_d_iput+0x63/0xbb [unionfs]
[<c0177811>] prune_dcache+0x151/0x1a0
[<c0177c4f>] shrink_dcache_memory+0x1f/0x50
[<c0149e45>] shrink_slab+0x155/0x190
[<c014b28f>] balance_pgdat+0x2ff/0x3d0
[<c0133120>] prepare_to_wait+0x20/0x70
[<c014b449>] kswapd+0xe9/0x120
[<c0133250>] autoremove_wake_function+0x0/0x60
[<c0133250>] autoremove_wake_function+0x0/0x60
[<c014b360>] kswapd+0x0/0x120
[<c0101485>] kernel_thread_helper+0x5/0x10
Code: 0b 3a d1 89 44 24 14 b8 a1 b2 39 d1 89 44 24 10 b8 5c 00 00 00 89
44 24 0c b8 e0 0c 3a d1 89 44 24 08 89 7c 24 04 e8a6 58 d8 ee <c6> 05 00
00 00 00 00 e9 69 fd ff ff 89 6c 24 18 be a0 13 3a d1
--
Christopher Huhn, GSI Darmstadt
'We apologise for the inconvenience'
_______________________________________________
unionfs mailing list
[email protected]
http://www.fsl.cs.sunysb.edu/mailman/listinfo/unionfs
