On Wed, Jun 05, 2013 at 04:28:33PM +0100, Martin wrote: > On 05/06/13 16:05, Hugo Mills wrote: > > On Wed, Jun 05, 2013 at 03:57:42PM +0100, Martin wrote: > >> Dear Devs, > >> > >> I have x4 4TB HDDs formatted with: > >> > >> mkfs.btrfs -L bu-16TB_0 -d raid1 -m raid1 /dev/sd[cdef] > >> > >> > >> /etc/fstab mounts with the options: > >> > >> noatime,noauto,space_cache,inode_cache > >> > >> > >> All on kernel 3.8.13. > >> > >> > >> Upon using rsync to copy some heavily hardlinked backups from > >> ReiserFS, I've seen: > >> > >> > >> The following "block rsv returned -28" is repeated 7 times until > >> there is a call trace for: > > > > This is ENOSPC. Can you post the output of "btrfs fi df > > /mountpoint" and "btrfs fi show", please? > > > btrfs fi df: > > Data, RAID1: total=2.85TB, used=2.84TB > Data: total=8.00MB, used=0.00 > System, RAID1: total=8.00MB, used=412.00KB > System: total=4.00MB, used=0.00 > Metadata, RAID1: total=27.00GB, used=25.82GB > Metadata: total=8.00MB, used=0.00 > > > btrfs fi show: > > Label: 'bu-16TB_0' uuid: 8fd9a0a8-9109-46db-8da0-396d9c6bc8e9 > Total devices 4 FS bytes used 2.87TB > devid 4 size 3.64TB used 1.44TB path /dev/sdf > devid 3 size 3.64TB used 1.44TB path /dev/sde > devid 1 size 3.64TB used 1.44TB path /dev/sdc > devid 2 size 3.64TB used 1.44TB path /dev/sdd
OK, so you've got plenty of space to allocate. There were some issues in this area (block reserves and ENOSPC, and I think specifically addressing the issue of ENOSPC when there's space available to allocate) that were fixed between 3.8 and 3.9 (and probably some between 3.9 and 3.10-rc as well), so upgrading your kernel _may_ help here. Something else that may possibly help as a sticking-plaster is to write metadata more slowly, so that you don't have quite so much of it waiting to be written out for the next transaction. Practically, this may involve things like running "sync" on a loop. But it's definitely a horrible hack that may help if you're desperate for a quick fix until you can finish creating metadata so quickly and upgrade your kernel... Hugo. > And df -h: > > Filesystem Size Used Avail Use% Mounted on > /dev/sde 15T 5.8T 8.9T 40% /mnt/sata16 > > > > > >> "WARNING: at fs/btrfs/super.c:256 > >> __btrfs_abort_transaction+0x3d/0xad()". > >> > >> Then, the mount is set read-only. > >> > >> > >> How to fix or debug? > >> > >> Thanks, Martin > >> > >> > >> > >> kernel: ------------[ cut here ]------------ kernel: WARNING: at > >> fs/btrfs/extent-tree.c:6372 btrfs_alloc_free_block+0xd3/0x29c() > >> kernel: Hardware name: GA-MA790FX-DS5 kernel: btrfs: block rsv > >> returned -28 kernel: Modules linked in: raid456 async_raid6_recov > >> async_memcpy async_pq async_xor xor async_tx raid6_pq act_police > >> cls_basic cls_flow cls_fw cls_u32 sch_tbf sch_prio sch_htb > >> sch_hfsc sch_ingress sch_sfq xt_CHECKSUM ipt_rpfilter > >> xt_statistic xt_CT xt_LOG xt_time xt_connlimit xt_realm > >> xt_addrtype xt_comment xt_recent xt_policy xt_nat ipt_ULOG > >> ipt_REJECT ipt_MASQUERADE ipt_ECN ipt_CLUSTERIP ipt_ah xt_set > >> ip_set nf_nat _tftp nf_nat_snmp_basic nf_conntrack_snmp > >> nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 > >> nf_nat_ftp nf_conntrack_tftp nf_conntrack_sip > >> nf_conntrack_proto_udplite nf_conntrack_proto_sctp > >> nf_conntrack_pptp nf_ conntrack_proto_gre nf_conntrack_netlink > >> nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc > >> nf_conntrack_h323 nf_conntrack_ftp xt_tcpmss xt_pkttype xt_owner > >> xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_mar k xt_mac > >> xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP > >> xt_dscp xt_dccp xt_conntrack xt_connmark xt_CLASSIFY xt_AUDIT > >> xt_tcpudp xt_state iptable_raw iptable_nat nf_nat_ipv4 nf_nat > >> nf_conntrack_ipv4 nf_defrag_i pv4 nf_conntrack iptable_mangle > >> nfnetlink iptable_filter ip_tables x_tables bridge stp llc rtc > >> snd_hda_codec_realtek fbcon bitblit softcursor font nouveau video > >> mxm_wmi cfbfillrect cfbimgblt cfbcopyarea i2c_algo_bit evdev d > >> rm_kms_helper snd_hda_intel ttm snd_hda_codec drm i2c_piix4 > >> pcspkr snd_pcm serio_raw snd_page_alloc snd_timer k8temp snd > >> i2c_core processor button thermal_sys sky2 wmi backlight fb fbdev > >> pata_acpi firewire_ohci firewire_cor e pata_atiixp usbhid > >> pata_jmicron sata_sil24 kernel: Pid: 10980, comm: btrfs-transacti > >> Not tainted 3.8.13-gentoo #1 kernel: Call Trace: kernel: > >> [<ffffffff811e6600>] ? btrfs_init_new_buffer+0xef/0xf6 kernel: > >> [<ffffffff810289c8>] ? warn_slowpath_common+0x78/0x8c kernel: > >> [<ffffffff81028a74>] ? warn_slowpath_fmt+0x45/0x4a kernel: > >> [<ffffffff81278f2c>] ? ___ratelimit+0xc4/0xd0 kernel: > >> [<ffffffff811e66da>] ? btrfs_alloc_free_block+0xd3/0x29c kernel: > >> [<ffffffff811d68e5>] ? __btrfs_cow_block+0x136/0x454 kernel: > >> [<ffffffff811f0d47>] ? btrfs_buffer_uptodate+0x40/0x56 kernel: > >> [<ffffffff811d6d8c>] ? btrfs_cow_block+0x132/0x19d kernel: > >> [<ffffffff811da606>] ? btrfs_search_slot+0x2f5/0x624 kernel: > >> [<ffffffff811dbc5a>] ? btrfs_insert_empty_items+0x5c/0xaf kernel: > >> [<ffffffff811e5089>] ? run_clustered_refs+0x852/0x8e6 kernel: > >> [<ffffffff811e4d20>] ? run_clustered_refs+0x4e9/0x8e6 kernel: > >> [<ffffffff811e7f6b>] ? btrfs_run_delayed_refs+0x10d/0x289 kernel: > >> [<ffffffff811f4ec6>] ? btrfs_commit_transaction+0x3a5/0x93c > >> kernel: [<ffffffff810427f0>] ? abort_exclusive_wait+0x79/0x79 > >> kernel: [<ffffffff811f5a8c>] ? start_transaction+0x311/0x408 > >> kernel: [<ffffffff811eed7e>] ? transaction_kthread+0xd1/0x16d > >> kernel: [<ffffffff811eecad>] ? btrfs_alloc_root+0x34/0x34 kernel: > >> [<ffffffff810420b3>] ? kthread+0xad/0xb5 kernel: > >> [<ffffffff81042006>] ? __kthread_parkme+0x5e/0x5e kernel: > >> [<ffffffff814315ac>] ? ret_from_fork+0x7c/0xb0 kernel: > >> [<ffffffff81042006>] ? __kthread_parkme+0x5e/0x5e kernel: ---[ > >> end trace b584e8ceb642293f ]--- kernel: ------------[ cut here > >> ]------------ > >> > >> > >> > >> kernel: ------------[ cut here ]------------ kernel: WARNING: at > >> fs/btrfs/super.c:256 __btrfs_abort_transaction+0x3d/0xad() > >> kernel: Hardware name: GA-MA790FX-DS5 kernel: btrfs: Transaction > >> aborted kernel: Modules linked in: raid456 async_raid6_recov > >> async_memcpy async_pq async_xor xor async_tx raid6_pq act_police > >> cls_basic cls_flow cls_fw cls_u32 sch_tbf sch_prio sch_htb > >> sch_hfsc sch_ingress sch_sfq xt_CHECKSUM ipt_rpfilter > >> xt_statistic xt_CT xt_LOG xt_time xt_connlimit xt_realm > >> xt_addrtype xt_comment xt_recent xt_policy xt_nat ipt_ULOG > >> ipt_REJECT ipt_MASQUERADE ipt_ECN ipt_CLUSTERIP ipt_ah xt_set > >> ip_set nf_nat _tftp nf_nat_snmp_basic nf_conntrack_snmp > >> nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 > >> nf_nat_ftp nf_conntrack_tftp nf_conntrack_sip > >> nf_conntrack_proto_udplite nf_conntrack_proto_sctp > >> nf_conntrack_pptp nf_ conntrack_proto_gre nf_conntrack_netlink > >> nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc > >> nf_conntrack_h323 nf_conntrack_ftp xt_tcpmss xt_pkttype xt_owner > >> xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_mar k xt_mac > >> xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP > >> xt_dscp xt_dccp xt_conntrack xt_connmark xt_CLASSIFY xt_AUDIT > >> xt_tcpudp xt_state iptable_raw iptable_nat nf_nat_ipv4 nf_nat > >> nf_conntrack_ipv4 nf_defrag_i pv4 nf_conntrack iptable_mangle > >> nfnetlink iptable_filter ip_tables x_tables bridge stp llc rtc > >> snd_hda_codec_realtek fbcon bitblit softcursor font nouveau video > >> mxm_wmi cfbfillrect cfbimgblt cfbcopyarea i2c_algo_bit evdev d > >> rm_kms_helper snd_hda_intel ttm snd_hda_codec drm i2c_piix4 > >> pcspkr snd_pcm serio_raw snd_page_alloc snd_timer k8temp snd > >> i2c_core processor button thermal_sys sky2 wmi backlight fb fbdev > >> pata_acpi firewire_ohci firewire_cor e pata_atiixp usbhid > >> pata_jmicron sata_sil24 kernel: Pid: 10980, comm: btrfs-transacti > >> Tainted: G W 3.8.13-gentoo #1 kernel: Call Trace: kernel: > >> [<ffffffff811d3600>] ? btrfs_printk+0x12/0xc2 kernel: > >> [<ffffffff810289c8>] ? warn_slowpath_common+0x78/0x8c kernel: > >> [<ffffffff81028a74>] ? warn_slowpath_fmt+0x45/0x4a kernel: > >> [<ffffffff811d5e00>] ? btrfs_release_path+0x5e/0x79 kernel: > >> [<ffffffff811d36ed>] ? __btrfs_abort_transaction+0x3d/0xad > >> kernel: [<ffffffff811ed97b>] ? btrfs_save_ino_cache+0x1d4/0x348 > >> kernel: [<ffffffff8142ce4c>] ? > >> commit_fs_roots.isra.25+0xa1/0x14a kernel: [<ffffffff81237a0f>] ? > >> btrfs_scrub_pause+0xd5/0xe4 kernel: [<ffffffff811f4f1a>] ? > >> btrfs_commit_transaction+0x3f9/0x93c kernel: [<ffffffff810427f0>] > >> ? abort_exclusive_wait+0x79/0x79 kernel: [<ffffffff811f5a8c>] ? > >> start_transaction+0x311/0x408 kernel: [<ffffffff811eed7e>] ? > >> transaction_kthread+0xd1/0x16d kernel: [<ffffffff811eecad>] ? > >> btrfs_alloc_root+0x34/0x34 kernel: [<ffffffff810420b3>] ? > >> kthread+0xad/0xb5 kernel: [<ffffffff81042006>] ? > >> __kthread_parkme+0x5e/0x5e kernel: [<ffffffff814315ac>] ? > >> ret_from_fork+0x7c/0xb0 kernel: [<ffffffff81042006>] ? > >> __kthread_parkme+0x5e/0x5e kernel: ---[ end trace > >> b584e8ceb6422945 ]--- kernel: BTRFS error (device sdf) in > >> btrfs_save_ino_cache:471: error 28 kernel: btrfs is forced > >> readonly kernel: BTRFS warning (device sdf): Skipping commit of > >> aborted transaction. kernel: BTRFS error (device sdf) in > >> cleanup_transaction:1391: error 28 > > > -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- All hope abandon, Ye who press Enter here. ---
signature.asc
Description: Digital signature
