On Sun, Nov 27, 2011 at 9:29 AM, Ted Yu <[email protected]> wrote: > We have many issues to tackle, build failure being one of them. > > Trying to reduce number of descriptors to 32k could be a goal for the > future. For the time being, its priority is not high.
My feeling is that this is a bug - the build failure is doing exactly what it's supposed to: alerting us of a bug. If the builds are using this many FDs, it's likely a leak -- just like the "out of memory" error last week was actually do to an infinite loop spewing exceptions in a test. Avoiding these errors by raising limits just hides the bug so we find it later in production. Unless we have a good explanation that shows that it's "just a test issue" we shouldn't ignore it or hide it by raising ulimit. -Todd > > On Sat, Nov 26, 2011 at 5:01 PM, lars hofhansl <[email protected]> wrote: > >> I agree. The tests should not need more than 32k open files. >> >> >> >> ----- Original Message ----- >> From: Todd Lipcon <[email protected]> >> To: [email protected] >> Cc: >> Sent: Saturday, November 26, 2011 10:42 AM >> Subject: Re: max number of open files when testing HBase on the Jenkins >> server >> >> I disagree that we should raise the limit. If we're using up 32k >> descriptors in unit tests, it's probably a real leak we should be >> concerned about. Maybe we can add a temporary patch upstream which >> does: >> catch (IOException ioe) { >> if (ioe.getMessage().contains("Too many open files")) { >> Runtime.getRuntime().exec("lsof -u " + System.getProperty("user.name >> ")); >> } >> } >> or something like that? >> >> -Todd >> >> On Thu, Nov 24, 2011 at 6:27 PM, Ted Yu <[email protected]> wrote: >> > Mikhail: >> > If you look at >> > https://builds.apache.org/job/PreCommit-HBASE-Build/364/console, you >> would >> > see: >> > >> > max memory size (kbytes, -m) unlimited >> > open files (-n) 32768 >> > >> > It is not clear why 32768 was not enough for TestAdmin. >> > >> > Normally we don't regard test failure resulted from 'Too many open files' >> > as real test failure. >> > >> > I agree the effective limit should be raised. >> > >> > Cheers >> > >> > On Thu, Nov 24, 2011 at 5:03 PM, Mikhail Bautin < >> > [email protected]> wrote: >> > >> >> Hello, >> >> >> >> I saw this error when testing my patch on Jenkins: >> >> >> >> Caused by: java.io.IOException: Too many open files >> >> at sun.nio.ch.IOUtil.initPipe(Native Method) >> >> at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:49) >> >> at >> >> >> sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18) >> >> at java.nio.channels.Selector.open(Selector.java:209) >> >> at >> >> >> org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:42) >> >> at sun.reflect.GeneratedConstructorAccessor40.newInstance(Unknown >> >> Source) >> >> at >> >> >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) >> >> at >> java.lang.reflect.Constructor.newInstance(Constructor.java:513) >> >> at java.lang.Class.newInstance0(Class.java:355) >> >> at java.lang.Class.newInstance(Class.java:308) >> >> at >> >> org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1737) >> >> ... 55 more >> >> >> >> (Full log: >> >> >> https://builds.apache.org/job/PreCommit-HBASE-Build/364//testReport/org.apache.hadoop.hbase.client/TestAdmin/testCheckHBaseAvailableClosesConnection/ >> >> ) >> >> >> >> I am not sure who maintains the Jenkins server, but it would be nice to >> >> increase the maximum number of open files for HBase unit tests. >> >> >> >> Thanks, >> >> --Mikhail >> >> >> > >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> >> > -- Todd Lipcon Software Engineer, Cloudera
