Here's the one I'm using for the test... [root@qauv2 zmopsx]# ls -l /data/traxnl3/trax2011/ERA.DET -rw-rw---- 1 perryt trax 123736145920 Feb 5 15:53 /data/traxnl3/trax2011/ERA.DET
So yeah.. they're pretty big. (There are others even bigger) Perry -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeffrey Butera Sent: Tuesday, February 05, 2013 3:51 PM To: U2 Users List Subject: Re: [U2] [UV] Large File Operations Kill Linux Perry I'm curious how large "large" is for you? Jeff Butera -- A tree falls the way it leans. Be careful which way you lean. The Lorax On Feb 5, 2013, at 5:45 PM, Perry Taylor <perry.tay...@zirmed.com> wrote: > I have engaged Redhat Support and it has already been escalated to their > Kernel team so at least it seems I have their attention :). I'll provide > updates as they become available. > > Perry > > -----Original Message----- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Dan Fitzgerald > Sent: Monday, February 04, 2013 3:32 PM > To: u2-users@listserver.u2ug.org > Subject: Re: [U2] [UV] Large File Operations Kill Linux > > > Other users could have been hanging at malloc. With a swappiness of 100 (on > some kernels) or 100 (on others) or "not 0 or 100"(not sure which behavior > you get on 2.6.18), pages wouldn't be getting freed up quickly enough duing > the creation/copying of a large file. > > Another thing to look at (although I prefer the support route, since you have > it), is /sys/kernel/mm/transparent_hugepage/defrag. Other people who have had > this problem alleviated it by setting this to "never". > > Of course, others fixed it by updating the kernel. My aged eyes read what you > have as 2.6.8.1... > >> Date: Mon, 4 Feb 2013 21:15:25 +0000 >> From: antli...@youngman.org.uk >> To: u2-users@listserver.u2ug.org >> Subject: Re: [U2] [UV] Large File Operations Kill Linux >> >> On 04/02/13 21:05, Dan Fitzgerald wrote: >>> >>> What's the value in /proc/sys/vm/swappiness? >> >> How will that make any difference? 2.6.18-348 SOUNDS like an ancient (in >> linux terms) kernel. Are you on RedHat support? >> >> This is a problem with the linux kernel that was addressed recently, >> iirc. Large amounts of io from a single process can swamp the queue, and >> the latest kernels have it fixed. >> >> If you've got RH support, see if you can find out if that's been >> backported into your kernel. >> >> Cheers, >> Wol >>> >>>> From: perry.tay...@zirmed.com >>>> To: u2-users@listserver.u2ug.org >>>> Date: Mon, 4 Feb 2013 20:53:13 +0000 >>>> Subject: Re: [U2] [UV] Large File Operations Kill Linux >>>> >>>> We're on RHEL5 (2.6.18-348.el5), ext3 and 132GB ram. >>>> >>>> -----Original Message----- >>>> From: u2-users-boun...@listserver.u2ug.org >>>> [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Symeon Breen >>>> Sent: Monday, February 04, 2013 9:23 AM >>>> To: 'U2 Users List' >>>> Subject: Re: [U2] [UV] Large File Operations Kill Linux >>>> >>>> A few questions - What linux version/distro are you on and what type of >>>> file system, and how much ram do you have >>>> >>>> -----Original Message----- >>>> From: u2-users-boun...@listserver.u2ug.org >>>> [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor >>>> Sent: 04 February 2013 15:57 >>>> To: U2-Users List >>>> Subject: [U2] [UV] Large File Operations Kill Linux >>>> >>>> Looking for some ideas on how to keep Linux from becoming largely >>>> unresponsive when creating large files. What happens is as the new file is >>>> being created the I/O buffer cache quickly fills up with dirty buffers. >>>> Until the kernel can flush these out to disk there is no avail buffers for >>>> I/O operations from other processes. . The most troubling manifestation >>>> of >>>> this is the transaction logging check point daemon gets *way* behind >>>> putting >>>> us as risk if we were to have a failure of some kind. >>>> >>>> I have tried using ionice and renice to slow the file creation down as much >>>> as possible. This help a little but is still a big problem. Any ideas how >>>> to get CREATE.FILE/RESIZE to play nice on Linux? >>>> >>>> Thanks. >>>> Perry >>>> Perry Taylor >>>> Senior MV Architect >>>> ZirMed >>>> 888 West Market Street, Suite 400 >>>> Louisville, KY 40202 >>>> www.zirmed.com<http://www.zirmed.com/> >> _______________________________________________ >> U2-Users mailing list >> U2-Users@listserver.u2ug.org >> http://listserver.u2ug.org/mailman/listinfo/u2-users > > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > > CONFIDENTIALITY NOTICE: This e-mail message, including any > attachments, is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is > prohibited. ZirMed, Inc. has strict policies regarding the > content of e-mail communications, specifically Protected Health > Information, any communications containing such material will > be returned to the originating party with such advisement > noted. If you are not the intended recipient, please contact > the sender by reply e-mail and destroy all copies of the > original message. > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users