Hi folks, recently we've been seeing more and more problems with bacula-fd messages in dmesg about a page allocation failure.
Platform is centos 6.2 64 bit, Version 5.2.6 compiled from Source using the stock distro gcc. We're using MariaDB 5.x as the db backend, here are some stats about the bacula installation itself: Total clients: 128 Total bytes stored: 77.33 TB Total files: 76905116 Database size: 111.72 GB The server has 18G RAM, backup performance is generally rather good. Online backups are going to disk (33TB full of 39TB, xfs based FS). Here's the message from dmesg: May 28 12:24:16 bacula-server kernel: bacula-sd: page allocation failure. order:1, mode:0x20 May 28 12:24:16 bacula-server kernel: Pid: 21923, comm: bacula-sd Not tainted 2.6.32-71.29.1.el6.x86_64 #1 May 28 12:24:16 bacula-server kernel: Call Trace: May 28 12:24:16 bacula-server kernel: <IRQ> [<ffffffff8111eab6>] __alloc_pages_nodemask+0x706/0x850 May 28 12:24:16 bacula-server kernel: [<ffffffff81156212>] kmem_getpages+0x62/0x170 May 28 12:24:16 bacula-server kernel: [<ffffffff81156e2a>] fallback_alloc+0x1ba/0x270 May 28 12:24:16 bacula-server kernel: [<ffffffff8115687f>] ? cache_grow+0x2cf/0x320 May 28 12:24:16 bacula-server kernel: [<ffffffff81156ba9>] ____cache_alloc_node+0x99/0x160 May 28 12:24:16 bacula-server kernel: [<ffffffff8115750b>] kmem_cache_alloc+0x11b/0x190 May 28 12:24:16 bacula-server kernel: [<ffffffff81404448>] sk_prot_alloc+0x48/0x180 May 28 12:24:16 bacula-server kernel: [<ffffffff81404692>] sk_clone+0x22/0x2a0 May 28 12:24:16 bacula-server kernel: [<ffffffff8144c276>] inet_csk_clone+0x16/0xd0 May 28 12:24:16 bacula-server kernel: [<ffffffff814651c3>] tcp_create_openreq_child+0x23/0x450 May 28 12:24:16 bacula-server kernel: [<ffffffff81462c0d>] tcp_v4_syn_recv_sock+0x4d/0x280 May 28 12:24:16 bacula-server kernel: [<ffffffff81464f81>] tcp_check_req+0x201/0x420 May 28 12:24:16 bacula-server kernel: [<ffffffff8146262b>] tcp_v4_do_rcv+0x35b/0x430 May 28 12:24:16 bacula-server kernel: [<ffffffff8105c484>] ? try_to_wake_up+0x284/0x380 May 28 12:24:16 bacula-server kernel: [<ffffffff81463e40>] tcp_v4_rcv+0x5b0/0x7e0 May 28 12:24:16 bacula-server kernel: [<ffffffff8105c592>] ? default_wake_function+0x12/0x20 May 28 12:24:16 bacula-server kernel: [<ffffffff81441e7d>] ip_local_deliver_finish+0xdd/0x2d0 May 28 12:24:16 bacula-server kernel: [<ffffffff81442108>] ip_local_deliver+0x98/0xa0 May 28 12:24:16 bacula-server kernel: [<ffffffff814415cd>] ip_rcv_finish+0x12d/0x440 May 28 12:24:16 bacula-server kernel: [<ffffffff81441b55>] ip_rcv+0x275/0x350 May 28 12:24:16 bacula-server kernel: [<ffffffff8140ffeb>] netif_receive_skb+0x38b/0x670 May 28 12:24:16 bacula-server kernel: [<ffffffff8126ce48>] ? is_swiotlb_buffer+0x18/0x50 May 28 12:24:16 bacula-server kernel: [<ffffffffa0269238>] bnx2_poll_work+0xd18/0x1240 [bnx2] May 28 12:24:16 bacula-server kernel: [<ffffffff8134a57a>] ? scsi_next_command+0x4a/0x60 May 28 12:24:16 bacula-server kernel: [<ffffffff8134b36e>] ? scsi_io_completion+0x35e/0x550 May 28 12:24:16 bacula-server kernel: [<ffffffff8105c846>] ? update_curr+0xe6/0x1e0 May 28 12:24:16 bacula-server kernel: [<ffffffffa026979d>] bnx2_poll_msix+0x3d/0xc0 [bnx2] May 28 12:24:16 bacula-server kernel: [<ffffffff81410b73>] net_rx_action+0x103/0x210 May 28 12:24:16 bacula-server kernel: [<ffffffff81073d67>] __do_softirq+0xb7/0x1e0 May 28 12:24:16 bacula-server kernel: [<ffffffff810d8a10>] ? handle_IRQ_event+0x60/0x170 May 28 12:24:16 bacula-server kernel: [<ffffffff81073dc4>] ? __do_softirq+0x114/0x1e0 May 28 12:24:16 bacula-server kernel: [<ffffffff810142cc>] call_softirq+0x1c/0x30 May 28 12:24:16 bacula-server kernel: [<ffffffff81015f35>] do_softirq+0x65/0xa0 May 28 12:24:16 bacula-server kernel: [<ffffffff81073b65>] irq_exit+0x85/0x90 May 28 12:24:16 bacula-server kernel: [<ffffffff814d0945>] do_IRQ+0x75/0xf0 May 28 12:24:16 bacula-server kernel: [<ffffffff81013ad3>] ret_from_intr+0x0/0x11 Any idea what's going wrong here? I don't see any significant swapping or memory usage when this happens. I can provide a full dmesg dump if that's helpful, I just didn't want to spam the list beyond measure this time. All the best & thanks in advance, Uwe -- NIONEX --- Ein Unternehmen der Bertelsmann AG ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
