Re: rsync on branches causes filesystem errors

[bl4]

sf...@users.sourceforge.net wrote:

How about other processes?
Are there any other process which writes to /, /rw or /ro?
Does your rsync write any log to somewhere in aufs?



This script is positioned after umountfs, before umountroot and halt. I 
think it's unlikely that any other processes are writing anything at 
that time. rsync is always run without --log-file.

Instead of running this script, I dropped to a shell and did something 
like this:
dmesg -c
echo 1 > /proc/sys/vm/block_dump
sleep 30
sync
sleep 30
sync
sleep 30
echo 0 > /proc/sys/vm/block_dump

If there were any write operations by other processes during this time, 
they should be listed in dmesg, but there were none.

Then I did:
echo 1 > /proc/sys/vm/block_dump
mount -o remount,rw /ro
rsync --exclude=".wh.*" --exclude=lost+found -aHSx --devices --specials 
/rw/ /ro/
mount -o remount,ro /ro
echo 0 > /proc/sys/vm/block_dump

There were some writes to hda1 (/ro), such as:
[  269.275225] rsync(2435): dirtied inode 2 (/) on hda1
[  269.275326] rsync(2435): dirtied inode 313873 (etc) on hda1
[  269.275445] rsync(2435): dirtied inode 43755 (cache) on hda1
etc. Among them were 3 writes to tmpfs (/rw):
[  269.276621] rsync(2434): dirtied inode 5933 (.cache) on tmpfs
[  269.303232] rsync(2434): dirtied inode 5349 (motd) on tmpfs
[  269.303276] rsync(2434): dirtied inode 4965 (sm-notify.pid) on tmpfs

I don't know if it means anything. I have attached full dmesg log.

My knowledge in this area is very limited so please tell me if you know 
other ways to investigate this situation.



I have FLUSH=ALL in /etc/defaults/aufs and I have /sbin/mount.aufs, 
which is enough according to the manpage.

If you don't run remount/umount aufs or auplink manually, pseudo-link
will not be flushed.



Okay, I have added auplink / flush


--
bl4
[  241.753731] EXT2-fs warning: mounting unchecked fs, running e2fsck is 
recommended
[  241.754944] exe(2433): WRITE block 0 on hda1
[  269.232380] busybox(2434): READ block 3832472 on hda1
[  269.234162] rsync(2434): READ block 3832568 on hda1
[  269.234812] rsync(2434): READ block 3833048 on hda1
[  269.236725] rsync(2434): READ block 1589544 on hda1
[  269.237539] rsync(2434): READ block 1589576 on hda1
[  269.239516] rsync(2434): READ block 3832912 on hda1
[  269.246729] rsync(2434): READ block 3832584 on hda1
[  269.249727] rsync(2434): READ block 3832840 on hda1
[  269.253222] rsync(2435): READ block 3832504 on hda1
[  269.253462] rsync(2435): READ block 3832512 on hda1
[  269.253483] rsync(2435): READ block 3832520 on hda1
[  269.253499] rsync(2435): READ block 3832528 on hda1
[  269.253511] rsync(2435): READ block 3832536 on hda1
[  269.253523] rsync(2435): READ block 3832544 on hda1
[  269.253535] rsync(2435): READ block 3832552 on hda1
[  269.253546] rsync(2435): READ block 3832560 on hda1
[  269.253562] rsync(2435): READ block 3832576 on hda1
[  269.275225] rsync(2435): dirtied inode 2 (/) on hda1
[  269.275326] rsync(2435): dirtied inode 313873 (etc) on hda1
[  269.275445] rsync(2435): dirtied inode 43755 (cache) on hda1
[  269.275461] rsync(2435): dirtied inode 314249 (run) on hda1
[  269.275785] rsync(2435): dirtied inode 160961 (tmp) on hda1
[  269.275797] rsync(2435): dirtied inode 161059 (.ICE-unix) on hda1
[  269.275804] rsync(2435): dirtied inode 160964 (.X11-unix) on hda1
[  269.275850] rsync(2435): dirtied inode 64537 (nfs) on hda1
[  269.275865] rsync(2435): dirtied inode 72446 (urandom) on hda1
[  269.275877] rsync(2435): dirtied inode 64386 (lock) on hda1
[  269.275886] rsync(2435): dirtied inode 98488 (lvm) on hda1
[  269.275892] rsync(2435): dirtied inode 72444 (log) on hda1
[  269.275926] rsync(2435): dirtied inode 64387 (run) on hda1
[  269.275961] rsync(2435): dirtied inode 92262 (avahi-daemon) on hda1
[  269.275968] rsync(2435): dirtied inode 98687 (network) on hda1
[  269.276418] rsync(2436): READ block 10223624 on hda1
[  269.276621] rsync(2434): dirtied inode 5933 (.cache) on tmpfs
[  269.277663] rsync(2436): dirtied inode 314258 (?) on hda1
[  269.278603] rsync(2436): READ block 10223616 on hda1
[  269.279713] rsync(2436): dirtied inode 320028 (adjtime) on hda1
[  269.279952] rsync(2436): dirtied inode 320026 (?) on hda1
[  269.280040] rsync(2436): dirtied inode 320029 (blkid.tab) on hda1
[  269.280114] rsync(2436): dirtied inode 320031 (?) on hda1
[  269.280169] rsync(2436): dirtied inode 320030 (blkid.tab.old) on hda1
[  269.280779] rsync(2436): dirtied inode 320030 (?) on hda1
[  269.280843] rsync(2436): dirtied inode 320025 (mtab) on hda1
[  269.280879] rsync(2436): dirtied inode 320025 (?) on hda1
[  269.280915] rsync(2436): dirtied inode 320027 (resolv.conf) on hda1
[  269.280946] rsync(2436): READ block 1313320 on hda1
[  269.281397] rsync(2436): dirtied inode 47217 (?) on hda1
[  269.281441] rsync(2436): READ block 1313312 on hda1
[  269.281654] rsync(2436): dirtied inode 43675 (.cache) on hda1
[  269.281757] rsync(2436): READ block 5505032 on hda1
[  269.282812] rsync(2436): dirtied inode 172323 (?) on hda1
[  269.282826] rsync(2436): dirtied inode 289729 (root) on hda1
[  269.282833] rsync(2436): dirtied inode 289729 (root) on hda1
[  269.282947] rsync(2436): READ block 5505024 on hda1
[  269.283307] rsync(2436): dirtied inode 172324 (.bash_history) on hda1
[  269.283350] rsync(2436): READ block 2097160 on hda1
[  269.283796] rsync(2436): dirtied inode 64396 (?) on hda1
[  269.283859] rsync(2436): dirtied inode 64552 (alsa) on hda1
[  269.283863] rsync(2436): dirtied inode 64552 (alsa) on hda1
[  269.283931] rsync(2436): READ block 2097152 on hda1
[  269.284119] rsync(2436): dirtied inode 65354 (asound.state) on hda1
[  269.284119] rsync(2436): READ block 2361896 on hda1
[  269.284119] rsync(2436): dirtied inode 72470 (?) on hda1
[  269.284119] rsync(2436): READ block 2361920 on hda1
[  269.284119] rsync(2436): dirtied inode 72963 (dhcp3) on hda1
[  269.284119] rsync(2436): dirtied inode 72963 (dhcp3) on hda1
[  269.284119] rsync(2436): READ block 2361888 on hda1
[  269.284119] rsync(2436): READ block 2621440 on hda1
[  269.284119] rsync(2436): dirtied inode 72449 (dhclient.eth0.leases) on hda1
[  269.284119] rsync(2436): dirtied inode 65330 (?) on hda1
[  269.284119] rsync(2436): dirtied inode 65355 (state) on hda1
[  269.284119] rsync(2436): dirtied inode 72868 (?) on hda1
[  269.284119] rsync(2436): READ block 2362120 on hda1
[  269.284119] rsync(2436): dirtied inode 76362 (random-seed) on hda1
[  269.284119] rsync(2436): dirtied inode 76362 (?) on hda1
[  269.284119] rsync(2436): dirtied inode 72454 (dmesg) on hda1
[  269.284119] rsync(2436): dirtied inode 76370 (?) on hda1
[  269.284119] rsync(2436): READ block 2363872 on hda1
[  269.284119] rsync(2436): dirtied inode 76366 (dmesg.0) on hda1
[  269.284119] rsync(2436): dirtied inode 76366 (?) on hda1
[  269.284119] rsync(2436): dirtied inode 72898 (dmesg.1.gz) on hda1
[  269.284119] rsync(2436): dirtied inode 72898 (?) on hda1
[  269.284119] rsync(2436): dirtied inode 72978 (dmesg.2.gz) on hda1
[  269.284119] rsync(2436): dirtied inode 72978 (?) on hda1
[  269.288584] rsync(2436): dirtied inode 73103 (dmesg.3.gz) on hda1
[  269.288635] rsync(2436): dirtied inode 73103 (?) on hda1
[  269.288767] rsync(2436): dirtied inode 76356 (dmesg.4.gz) on hda1
[  269.288789] rsync(2436): dirtied inode 76356 (?) on hda1
[  269.298065] rsync(2436): READ block 3145728 on hda1
[  269.303232] rsync(2434): dirtied inode 5349 (motd) on tmpfs
[  269.303276] rsync(2434): dirtied inode 4965 (sm-notify.pid) on tmpfs
[  269.303841] rsync(2436): dirtied inode 72970 (wtmp) on hda1
[  269.304348] rsync(2436): dirtied inode 72970 (?) on hda1
[  269.304360] rsync(2436): dirtied inode 72445 (fsck) on hda1
[  269.304364] rsync(2436): dirtied inode 72445 (fsck) on hda1
[  269.304482] rsync(2436): dirtied inode 76083 (checkfs) on hda1
[  269.304503] rsync(2436): dirtied inode 65355 (?) on hda1
[  269.304536] rsync(2436): dirtied inode 65079 (motd) on hda1
[  269.304560] rsync(2436): dirtied inode 65079 (?) on hda1
[  269.304625] rsync(2436): dirtied inode 65080 (portmap.pid) on hda1
[  269.304645] rsync(2436): dirtied inode 65080 (?) on hda1
[  269.304752] rsync(2436): dirtied inode 65081 (portmap.state) on hda1
[  269.304772] rsync(2436): dirtied inode 65081 (?) on hda1
[  269.304876] rsync(2436): dirtied inode 65082 (portmap_mapping) on hda1
[  269.304895] rsync(2436): dirtied inode 65082 (?) on hda1
[  269.304921] rsync(2436): dirtied inode 65084 (rpc.statd.pid) on hda1
[  269.304941] rsync(2436): dirtied inode 65084 (?) on hda1
[  269.305044] rsync(2436): dirtied inode 65139 (sm-notify.pid) on hda1
[  269.305065] rsync(2436): dirtied inode 65139 (?) on hda1
[  269.305193] rsync(2436): dirtied inode 65257 (utmp) on hda1
[  272.925972] exe(2437): WRITE block 10321920 on hda1
[  272.926043] exe(2437): WRITE block 1442000 on hda1
[  272.926215] exe(2437): WRITE block 2355080 on hda1
[  272.926281] exe(2437): WRITE block 2539536 on hda1
[  272.926589] exe(2437): WRITE block 2408528 on hda1
[  272.926615] exe(2437): WRITE block 2113536 on hda1
[  272.927787] exe(2437): WRITE block 10291408 on hda1
[  272.927863] exe(2437): WRITE block 10291480 on hda1
[  272.927891] exe(2437): WRITE block 10291488 on hda1
[  272.927933] exe(2437): WRITE block 10291552 on hda1
[  272.927974] exe(2437): WRITE block 10291560 on hda1
[  272.927976] exe(2437): WRITE block 1379760 on hda1
[  272.928436] exe(2437): WRITE block 5627160 on hda1
[  272.928469] exe(2437): WRITE block 9486336 on hda1
[  272.928532] exe(2437): WRITE block 2169088 on hda1
[  272.928583] exe(2437): WRITE block 2169168 on hda1
[  272.928629] exe(2437): WRITE block 2355200 on hda1
[  272.928657] exe(2437): WRITE block 2775920 on hda1
[  272.928680] exe(2437): WRITE block 2368560 on hda1
[  272.928712] exe(2437): WRITE block 2169176 on hda1
[  272.928820] exe(2437): WRITE block 2775928 on hda1
[  272.928873] exe(2437): WRITE block 2775936 on hda1
[  272.928904] exe(2437): WRITE block 2775976 on hda1
[  272.928932] exe(2437): WRITE block 2776016 on hda1
[  272.928957] exe(2437): WRITE block 2562872 on hda1
[  272.928974] exe(2437): WRITE block 2578360 on hda1
[  272.929016] exe(2437): WRITE block 2584136 on hda1
[  272.929058] exe(2437): WRITE block 2584200 on hda1
[  272.929189] exe(2437): WRITE block 2776032 on hda1
[  272.929317] exe(2437): WRITE block 2776128 on hda1
[  272.929366] exe(2437): WRITE block 2776136 on hda1
[  272.929483] exe(2437): WRITE block 2777216 on hda1
[  272.929567] exe(2437): WRITE block 2777472 on hda1
[  272.929600] exe(2437): WRITE block 2778112 on hda1
[  272.929620] exe(2437): WRITE block 2778216 on hda1
[  272.929648] exe(2437): WRITE block 2779584 on hda1
[  272.929668] exe(2437): WRITE block 2780608 on hda1
[  272.929716] exe(2437): WRITE block 2783872 on hda1
[  272.929736] exe(2437): WRITE block 3200864 on hda1
[  272.929753] exe(2437): WRITE block 3250096 on hda1
[  272.929777] exe(2437): WRITE block 3250120 on hda1
[  272.929802] exe(2437): WRITE block 3250184 on hda1
[  272.929847] exe(2437): WRITE block 3250504 on hda1
[  272.929871] exe(2437): WRITE block 3250600 on hda1
[  272.929925] exe(2437): WRITE block 3251456 on hda1
[  272.930014] exe(2437): WRITE block 3251712 on hda1
[  272.930042] exe(2437): WRITE block 2772264 on hda1
[  272.930083] exe(2437): WRITE block 2539528 on hda1
[  272.930112] exe(2437): WRITE block 2255488 on hda1
[  272.930166] exe(2437): WRITE block 2255696 on hda1
[  272.930211] exe(2437): WRITE block 2772272 on hda1
[  272.930233] exe(2437): WRITE block 2772280 on hda1
[  272.930270] exe(2437): WRITE block 2772288 on hda1
[  272.930289] exe(2437): WRITE block 2772296 on hda1
[  272.930648] exe(2437): WRITE block 0 on hda1
[  272.930772] exe(2437): WRITE block 8 on hda1
[  272.930792] exe(2437): WRITE block 2608 on hda1
[  272.930810] exe(2437): WRITE block 1313312 on hda1
[  272.930827] exe(2437): WRITE block 1313320 on hda1
[  272.930842] exe(2437): WRITE block 1315040 on hda1
[  272.930858] exe(2437): WRITE block 1315080 on hda1
[  272.930897] exe(2437): WRITE block 1316816 on hda1
[  272.930915] exe(2437): WRITE block 2097152 on hda1
[  272.930931] exe(2437): WRITE block 2097160 on hda1
[  272.930946] exe(2437): WRITE block 2097168 on hda1
[  272.930959] exe(2437): WRITE block 2097240 on hda1
[  272.930978] exe(2437): WRITE block 2097248 on hda1
[  272.930992] exe(2437): WRITE block 2097512 on hda1
[  272.931014] exe(2437): WRITE block 2097544 on hda1
[  272.931031] exe(2437): WRITE block 2097600 on hda1
[  272.931047] exe(2437): WRITE block 2097640 on hda1
[  272.931064] exe(2437): WRITE block 2097648 on hda1
[  272.931078] exe(2437): WRITE block 2361888 on hda1
[  272.931093] exe(2437): WRITE block 2361896 on hda1
[  272.931106] exe(2437): WRITE block 2361904 on hda1
[  272.931119] exe(2437): WRITE block 2361912 on hda1
[  272.931132] exe(2437): WRITE block 2361920 on hda1
[  272.931680] exe(2437): WRITE block 2362120 on hda1
[  272.931704] exe(2437): WRITE block 2362136 on hda1
[  272.931720] exe(2437): WRITE block 2362168 on hda1
[  272.931737] exe(2437): WRITE block 2362176 on hda1
[  272.931752] exe(2437): WRITE block 2362232 on hda1
[  272.931792] exe(2437): WRITE block 2363728 on hda1
[  272.931808] exe(2437): WRITE block 2363864 on hda1
[  272.931824] exe(2437): WRITE block 2363872 on hda1
[  272.931838] exe(2437): WRITE block 2621440 on hda1
[  272.934550] exe(2437): WRITE block 2885464 on hda1
[  272.934584] exe(2437): WRITE block 3145728 on hda1
[  272.934604] exe(2437): WRITE block 3146696 on hda1
[  272.934622] exe(2437): WRITE block 3146792 on hda1
[  272.934639] exe(2437): WRITE block 5242896 on hda1
[  272.934657] exe(2437): WRITE block 5242944 on hda1
[  272.934674] exe(2437): WRITE block 5505024 on hda1
[  272.934733] exe(2437): WRITE block 5505032 on hda1
[  272.934749] exe(2437): WRITE block 5506696 on hda1
[  272.934779] exe(2437): WRITE block 9437200 on hda1
[  272.934796] exe(2437): WRITE block 10223616 on hda1
[  272.934811] exe(2437): WRITE block 10223624 on hda1
[  272.934825] exe(2437): WRITE block 10223632 on hda1
[  272.934838] exe(2437): WRITE block 10223816 on hda1
[  272.934854] exe(2437): WRITE block 10223824 on hda1
[  272.934868] exe(2437): WRITE block 10226704 on hda1
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev