Thanks Grant. I can't easily upgrade the Hadoop cluster that I'm using so I figured the mailing list was a good shot to see if anybody else had run into the same problem. If not (and that seems to be the case), then I really do need to go bug the cluster maintainer to upgrade.
Anyway, in case somebody has actually seen this, and for completeness, here's the output from lsof. COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME java 19625 mdesnoyer cwd DIR 8,1 4096 4138417 /home/mdesnoyer/src/graffiti0/summer09/hadoop java 19625 mdesnoyer rtd DIR 8,1 4096 2 / java 19625 mdesnoyer txt REG 8,1 47308 6645980 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/bin/java java 19625 mdesnoyer mem REG 8,1 83552 2556065 /lib/libz.so.1.2.3.3 java 19625 mdesnoyer mem REG 8,1 1145980 6646075 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/resources.jar java 19625 mdesnoyer mem REG 8,1 102006 5243603 /opt/hadoop-0.18.3_dumbo/lib/native/Linux-i386-32/libhadoop.so java 19625 mdesnoyer mem REG 8,1 75472 2573671 /lib/tls/i686/cmov/libresolv-2.9.so java 19625 mdesnoyer mem REG 8,1 21976 2573658 /lib/tls/i686/cmov/libnss_dns-2.9.so java 19625 mdesnoyer mem REG 8,1 36913 6646018 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/libnio.so java 19625 mdesnoyer mem REG 8,1 33900 6646019 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/libmanagement.so java 19625 mdesnoyer mem REG 8,1 95801 6646017 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/libnet.so java 19625 mdesnoyer mem REG 8,1 2145163 4138922 /home/mdesnoyer/src/graffiti0/summer09/hadoop/lda-mapreduce.jar java 19625 mdesnoyer mem REG 8,1 50493 5243584 /opt/hadoop-0.18.3_dumbo/lib/jetty-ext/jsp-api.jar java 19625 mdesnoyer mem REG 8,1 105502 5243582 /opt/hadoop-0.18.3_dumbo/lib/jetty-ext/jasper-runtime.jar java 19625 mdesnoyer mem REG 8,1 351903 5243585 /opt/hadoop-0.18.3_dumbo/lib/jetty-ext/jasper-compiler.jar java 19625 mdesnoyer mem REG 8,1 112341 5243583 /opt/hadoop-0.18.3_dumbo/lib/jetty-ext/commons-el.jar java 19625 mdesnoyer mem REG 8,1 15010 5243591 /opt/hadoop-0.18.3_dumbo/lib/xmlenc-0.52.jar java 19625 mdesnoyer mem REG 8,1 8601 5243613 /opt/hadoop-0.18.3_dumbo/lib/slf4j-log4j12-1.4.3.jar java 19625 mdesnoyer mem REG 8,1 15345 5243588 /opt/hadoop-0.18.3_dumbo/lib/slf4j-api-1.4.3.jar java 19625 mdesnoyer mem REG 8,1 97689 5243586 /opt/hadoop-0.18.3_dumbo/lib/servlet-api.jar java 19625 mdesnoyer mem REG 8,1 65261 5243614 /opt/hadoop-0.18.3_dumbo/lib/oro-2.0.8.jar java 19625 mdesnoyer mem REG 8,1 391834 5243599 /opt/hadoop-0.18.3_dumbo/lib/log4j-1.2.15.jar java 19625 mdesnoyer mem REG 8,1 9484 5243580 /opt/hadoop-0.18.3_dumbo/lib/kfs-0.1.3.jar java 19625 mdesnoyer mem REG 8,1 121070 5243616 /opt/hadoop-0.18.3_dumbo/lib/junit-3.8.1.jar java 19625 mdesnoyer mem REG 8,1 665638 5243592 /opt/hadoop-0.18.3_dumbo/lib/jetty-5.1.4.jar java 19625 mdesnoyer mem REG 8,1 288534 5243598 /opt/hadoop-0.18.3_dumbo/lib/jets3t-0.6.0.jar java 19625 mdesnoyer mem REG 8,1 180792 5243589 /opt/hadoop-0.18.3_dumbo/lib/commons-net-1.4.1.jar java 19625 mdesnoyer mem REG 8,1 26202 5243590 /opt/hadoop-0.18.3_dumbo/lib/commons-logging-api-1.0.4.jar java 19625 mdesnoyer mem REG 8,1 38015 5243594 /opt/hadoop-0.18.3_dumbo/lib/commons-logging-1.0.4.jar java 19625 mdesnoyer mem REG 8,1 279781 5243597 /opt/hadoop-0.18.3_dumbo/lib/commons-httpclient-3.0.1.jar java 19625 mdesnoyer mem REG 8,1 46725 5243617 /opt/hadoop-0.18.3_dumbo/lib/commons-codec-1.3.jar java 19625 mdesnoyer mem REG 8,1 258337 5243596 /opt/hadoop-0.18.3_dumbo/lib/commons-cli-2.0-SNAPSHOT.jar java 19625 mdesnoyer mem REG 8,1 1985528 3599303 /opt/hadoop-0.18.3_dumbo/hadoop-0.18.3_dumbo-core.jar java 19625 mdesnoyer mem REG 8,1 12588713 6645952 /usr/lib/jvm/java-6-sun-1.6.0.14/lib/tools.jar java 19625 mdesnoyer mem REG 8,1 256316 6276046 /usr/lib/locale/en_US.utf8/LC_CTYPE java 19625 mdesnoyer mem REG 8,1 51669409 6646241 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/rt.jar java 19625 mdesnoyer mem REG 8,1 618950 6646646 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/jsse.jar java 19625 mdesnoyer mem REG 8,1 76348 6646013 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/libzip.so java 19625 mdesnoyer mem REG 8,1 42504 2573660 /lib/tls/i686/cmov/libnss_files-2.9.so java 19625 mdesnoyer mem REG 8,1 38444 2573664 /lib/tls/i686/cmov/libnss_nis-2.9.so java 19625 mdesnoyer mem REG 8,1 30436 2573656 /lib/tls/i686/cmov/libnss_compat-2.9.so java 19625 mdesnoyer mem REG 8,1 26048 5652723 /usr/lib/gconv/gconv-modules.cache java 19625 mdesnoyer mem REG 8,1 32768 3589971 /tmp/hsperfdata_mdesnoyer/19625 java 19625 mdesnoyer mem REG 8,1 87804 2573654 /lib/tls/i686/cmov/libnsl-2.9.so java 19625 mdesnoyer mem REG 8,1 38694 6645998 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/native_threads/libhpi.so java 19625 mdesnoyer mem REG 8,1 189159 6646010 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/libjava.so java 19625 mdesnoyer mem REG 8,1 30624 2573673 /lib/tls/i686/cmov/librt-2.9.so java 19625 mdesnoyer mem REG 8,1 149328 2573651 /lib/tls/i686/cmov/libm-2.9.so java 19625 mdesnoyer mem REG 8,1 9180183 6646000 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/server/libjvm.so java 19625 mdesnoyer mem REG 8,1 1442180 2573643 /lib/tls/i686/cmov/libc-2.9.so java 19625 mdesnoyer mem REG 8,1 9676 2573649 /lib/tls/i686/cmov/libdl-2.9.so java 19625 mdesnoyer mem REG 8,1 38142 6646012 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/jli/libjli.so java 19625 mdesnoyer mem REG 8,1 116405 2573669 /lib/tls/i686/cmov/libpthread-2.9.so java 19625 mdesnoyer mem REG 8,1 7552 2555999 /lib/libnss_mdns4_minimal.so.2 java 19625 mdesnoyer mem REG 8,1 56701 6646009 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/i386/libverify.so java 19625 mdesnoyer mem REG 8,1 117348 2555927 /lib/ ld-2.9.so java 19625 mdesnoyer 0u CHR 136,0 2 /dev/pts/0 java 19625 mdesnoyer 1u CHR 136,0 2 /dev/pts/0 java 19625 mdesnoyer 2u CHR 136,0 2 /dev/pts/0 java 19625 mdesnoyer 3r REG 8,1 51669409 6646241 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/rt.jar java 19625 mdesnoyer 4r REG 8,1 12588713 6645952 /usr/lib/jvm/java-6-sun-1.6.0.14/lib/tools.jar java 19625 mdesnoyer 5r REG 8,1 1985528 3599303 /opt/hadoop-0.18.3_dumbo/hadoop-0.18.3_dumbo-core.jar java 19625 mdesnoyer 6r REG 8,1 258337 5243596 /opt/hadoop-0.18.3_dumbo/lib/commons-cli-2.0-SNAPSHOT.jar java 19625 mdesnoyer 7r REG 8,1 46725 5243617 /opt/hadoop-0.18.3_dumbo/lib/commons-codec-1.3.jar java 19625 mdesnoyer 8r REG 8,1 279781 5243597 /opt/hadoop-0.18.3_dumbo/lib/commons-httpclient-3.0.1.jar java 19625 mdesnoyer 9r REG 8,1 38015 5243594 /opt/hadoop-0.18.3_dumbo/lib/commons-logging-1.0.4.jar java 19625 mdesnoyer 10r REG 8,1 26202 5243590 /opt/hadoop-0.18.3_dumbo/lib/commons-logging-api-1.0.4.jar java 19625 mdesnoyer 11r REG 8,1 180792 5243589 /opt/hadoop-0.18.3_dumbo/lib/commons-net-1.4.1.jar java 19625 mdesnoyer 12r REG 8,1 288534 5243598 /opt/hadoop-0.18.3_dumbo/lib/jets3t-0.6.0.jar java 19625 mdesnoyer 13r REG 8,1 665638 5243592 /opt/hadoop-0.18.3_dumbo/lib/jetty-5.1.4.jar java 19625 mdesnoyer 14r REG 8,1 121070 5243616 /opt/hadoop-0.18.3_dumbo/lib/junit-3.8.1.jar java 19625 mdesnoyer 15r REG 8,1 9484 5243580 /opt/hadoop-0.18.3_dumbo/lib/kfs-0.1.3.jar java 19625 mdesnoyer 16r REG 8,1 391834 5243599 /opt/hadoop-0.18.3_dumbo/lib/log4j-1.2.15.jar java 19625 mdesnoyer 17r REG 8,1 65261 5243614 /opt/hadoop-0.18.3_dumbo/lib/oro-2.0.8.jar java 19625 mdesnoyer 18r REG 8,1 97689 5243586 /opt/hadoop-0.18.3_dumbo/lib/servlet-api.jar java 19625 mdesnoyer 19r REG 8,1 15345 5243588 /opt/hadoop-0.18.3_dumbo/lib/slf4j-api-1.4.3.jar java 19625 mdesnoyer 20r REG 8,1 8601 5243613 /opt/hadoop-0.18.3_dumbo/lib/slf4j-log4j12-1.4.3.jar java 19625 mdesnoyer 21r REG 8,1 15010 5243591 /opt/hadoop-0.18.3_dumbo/lib/xmlenc-0.52.jar java 19625 mdesnoyer 22r REG 8,1 112341 5243583 /opt/hadoop-0.18.3_dumbo/lib/jetty-ext/commons-el.jar java 19625 mdesnoyer 23r REG 8,1 351903 5243585 /opt/hadoop-0.18.3_dumbo/lib/jetty-ext/jasper-compiler.jar java 19625 mdesnoyer 24r REG 8,1 105502 5243582 /opt/hadoop-0.18.3_dumbo/lib/jetty-ext/jasper-runtime.jar java 19625 mdesnoyer 25r REG 8,1 50493 5243584 /opt/hadoop-0.18.3_dumbo/lib/jetty-ext/jsp-api.jar java 19625 mdesnoyer 26r CHR 1,8 1817 /dev/random java 19625 mdesnoyer 27r CHR 1,9 1825 /dev/urandom java 19625 mdesnoyer 28r REG 8,1 2145163 4138922 /home/mdesnoyer/src/graffiti0/summer09/hadoop/lda-mapreduce.jar java 19625 mdesnoyer 29r REG 8,1 618950 6646646 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/jsse.jar java 19625 mdesnoyer 30r REG 8,1 1145980 6646075 /usr/lib/jvm/java-6-sun-1.6.0.14/jre/lib/resources.jar java 19625 mdesnoyer 31r FIFO 0,6 1344679 pipe java 19625 mdesnoyer 32u sock 0,4 1344677 can't identify protocol java 19625 mdesnoyer 33w FIFO 0,6 1344679 pipe java 19625 mdesnoyer 34u 0000 0,7 0 32 anon_inode java 19625 mdesnoyer 35u unix 0xe280c700 1344680 socket java 19625 mdesnoyer 36r IPv6 1348265 TCP luna.corn.cnwk:42903->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 37w IPv6 1406254 TCP luna.corn.cnwk:49999->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 38u IPv6 1407766 TCP luna.corn.cnwk:51609->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 40u IPv6 1407979 TCP luna.corn.cnwk:35437->c17-dw-dev50-hdfs-dn-n3.corn.com:50031 (ESTABLISHED) java 19625 mdesnoyer 41u IPv6 1347947 TCP luna.corn.cnwk:39687->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 42r IPv6 1347948 TCP luna.corn.cnwk:35215->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 43w IPv6 1347949 TCP luna.corn.cnwk:35216->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 44u IPv6 1347950 TCP luna.corn.cnwk:35217->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 48u IPv6 1347976 TCP luna.corn.cnwk:35218->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 49u IPv6 1347977 TCP luna.corn.cnwk:35219->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 50u IPv6 1347978 TCP luna.corn.cnwk:42880->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 51u IPv6 1348006 TCP luna.corn.cnwk:35221->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 52u IPv6 1348007 TCP luna.corn.cnwk:42882->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 53u IPv6 1348034 TCP luna.corn.cnwk:35223->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 54u IPv6 1348035 TCP luna.corn.cnwk:35224->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 55u IPv6 1348036 TCP luna.corn.cnwk:42885->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 56u IPv6 1348064 TCP luna.corn.cnwk:35226->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 57u IPv6 1348065 TCP luna.corn.cnwk:42887->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 58u IPv6 1348092 TCP luna.corn.cnwk:39701->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 59u IPv6 1348093 TCP luna.corn.cnwk:35229->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 60u IPv6 1348094 TCP luna.corn.cnwk:42890->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 61u IPv6 1348122 TCP luna.corn.cnwk:55555->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 62u IPv6 1348123 TCP luna.corn.cnwk:35232->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 63u IPv6 1348150 TCP luna.corn.cnwk:39706->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 64u IPv6 1348151 TCP luna.corn.cnwk:35234->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 65u IPv6 1348178 TCP luna.corn.cnwk:42895->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 66u IPv6 1348179 TCP luna.corn.cnwk:55560->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 67u IPv6 1348188 TCP luna.corn.cnwk:42897->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 68u IPv6 1348208 TCP luna.corn.cnwk:42898->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 69u IPv6 1348209 TCP luna.corn.cnwk:55563->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 70u IPv6 1348236 TCP luna.corn.cnwk:55564->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 71u IPv6 1348237 TCP luna.corn.cnwk:55565->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 72u IPv6 1348264 TCP luna.corn.cnwk:35242->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 73u IPv6 1348266 TCP luna.corn.cnwk:35244->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 74u IPv6 1348293 TCP luna.corn.cnwk:39718->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 75u IPv6 1348294 TCP luna.corn.cnwk:35246->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 76u IPv6 1348321 TCP luna.corn.cnwk:42907->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 77u IPv6 1348322 TCP luna.corn.cnwk:42908->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 78u IPv6 1348325 TCP luna.corn.cnwk:35249->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 79u IPv6 1348351 TCP luna.corn.cnwk:42910->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 80u IPv6 1348352 TCP luna.corn.cnwk:39724->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 81u IPv6 1348379 TCP luna.corn.cnwk:39725->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 82u IPv6 1348380 TCP luna.corn.cnwk:55577->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 83u IPv6 1348407 TCP luna.corn.cnwk:39727->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 84u IPv6 1348408 TCP luna.corn.cnwk:39728->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 85u IPv6 1348409 TCP luna.corn.cnwk:35256->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 86u IPv6 1348410 TCP luna.corn.cnwk:34784->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 88r IPv6 1406224 TCP luna.corn.cnwk:56589->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 89u IPv6 1406225 TCP luna.corn.cnwk:45601->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 90r IPv6 1406226 TCP luna.corn.cnwk:45602->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 91w IPv6 1406227 TCP luna.corn.cnwk:45603->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 92u IPv6 1406228 TCP luna.corn.cnwk:49973->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 94w IPv6 1407722 TCP luna.corn.cnwk:48650->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 95u IPv6 1407734 TCP luna.corn.cnwk:48652->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 96u IPv6 1406229 TCP luna.corn.cnwk:54989->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 97u IPv6 1406230 TCP luna.corn.cnwk:56595->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 98u IPv6 1406231 TCP luna.corn.cnwk:49976->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 99u IPv6 1406232 TCP luna.corn.cnwk:49977->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 100u IPv6 1406233 TCP luna.corn.cnwk:45609->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 101u IPv6 1406234 TCP luna.corn.cnwk:56599->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 102u IPv6 1406235 TCP luna.corn.cnwk:45611->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 103u IPv6 1406236 TCP luna.corn.cnwk:54996->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 104u IPv6 1406237 TCP luna.corn.cnwk:45613->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 105u IPv6 1406238 TCP luna.corn.cnwk:56603->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 106u IPv6 1406239 TCP luna.corn.cnwk:56604->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 107u IPv6 1406240 TCP luna.corn.cnwk:45616->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 108u IPv6 1406241 TCP luna.corn.cnwk:45617->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 109u IPv6 1406242 TCP luna.corn.cnwk:49987->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 110u IPv6 1406243 TCP luna.corn.cnwk:49988->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 111u IPv6 1406244 TCP luna.corn.cnwk:56609->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 112u IPv6 1406245 TCP luna.corn.cnwk:45621->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 113u IPv6 1406246 TCP luna.corn.cnwk:45622->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 114u IPv6 1406247 TCP luna.corn.cnwk:55007->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 115u IPv6 1406248 TCP luna.corn.cnwk:45624->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 116u IPv6 1406249 TCP luna.corn.cnwk:55009->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 117u IPv6 1406250 TCP luna.corn.cnwk:56615->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 118u IPv6 1406251 TCP luna.corn.cnwk:55011->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 119u IPv6 1406252 TCP luna.corn.cnwk:56617->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 120u IPv6 1406253 TCP luna.corn.cnwk:55013->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 121u IPv6 1406255 TCP luna.corn.cnwk:45631->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 122u IPv6 1406256 TCP luna.corn.cnwk:56621->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 123u IPv6 1406257 TCP luna.corn.cnwk:45633->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 124u IPv6 1406258 TCP luna.corn.cnwk:50003->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 125u IPv6 1406259 TCP luna.corn.cnwk:45635->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 126u IPv6 1406260 TCP luna.corn.cnwk:56625->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 127u IPv6 1406261 TCP luna.corn.cnwk:45637->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 128u IPv6 1406262 TCP luna.corn.cnwk:55022->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 129u IPv6 1406263 TCP luna.corn.cnwk:45639->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 130u IPv6 1406264 TCP luna.corn.cnwk:56629->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 131u IPv6 1406265 TCP luna.corn.cnwk:50010->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 132u IPv6 1406266 TCP luna.corn.cnwk:56631->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 133u IPv6 1406267 TCP luna.corn.cnwk:50012->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 134u IPv6 1407738 TCP luna.corn.cnwk:42171->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 135u IPv6 1407739 TCP luna.corn.cnwk:42172->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 136u IPv6 1407740 TCP luna.corn.cnwk:48658->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 137u IPv6 1407741 TCP luna.corn.cnwk:42174->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 138u IPv6 1407742 TCP luna.corn.cnwk:51586->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 139u IPv6 1407743 TCP luna.corn.cnwk:48661->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 143u IPv6 1407744 TCP luna.corn.cnwk:51588->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 144u IPv6 1407745 TCP luna.corn.cnwk:42178->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 145u IPv6 1407746 TCP luna.corn.cnwk:54228->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 146u IPv6 1407747 TCP luna.corn.cnwk:51591->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 147u IPv6 1407748 TCP luna.corn.cnwk:48666->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 148u IPv6 1407749 TCP luna.corn.cnwk:48667->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 149u IPv6 1407750 TCP luna.corn.cnwk:42183->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 150u IPv6 1407751 TCP luna.corn.cnwk:51595->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 151u IPv6 1407752 TCP luna.corn.cnwk:54234->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 152u IPv6 1407753 TCP luna.corn.cnwk:48671->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 153u IPv6 1407754 TCP luna.corn.cnwk:48672->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 154u IPv6 1407755 TCP luna.corn.cnwk:42188->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 155u IPv6 1407756 TCP luna.corn.cnwk:54238->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 156u IPv6 1407757 TCP luna.corn.cnwk:42190->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 157u IPv6 1407758 TCP luna.corn.cnwk:51602->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 158u IPv6 1407759 TCP luna.corn.cnwk:48677->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 159u IPv6 1407760 TCP luna.corn.cnwk:42193->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 160u IPv6 1407761 TCP luna.corn.cnwk:54243->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 161u IPv6 1407762 TCP luna.corn.cnwk:54244->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 162u IPv6 1407763 TCP luna.corn.cnwk:42196->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 163u IPv6 1407764 TCP luna.corn.cnwk:54246->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 164u IPv6 1407767 TCP luna.corn.cnwk:51610->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 165u IPv6 1407768 TCP luna.corn.cnwk:42200->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 166u IPv6 1407769 TCP luna.corn.cnwk:51612->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 167u IPv6 1407770 TCP luna.corn.cnwk:51613->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 168u IPv6 1407771 TCP luna.corn.cnwk:48688->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 169u IPv6 1407772 TCP luna.corn.cnwk:42204->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 170u IPv6 1407773 TCP luna.corn.cnwk:42205->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 171u IPv6 1407774 TCP luna.corn.cnwk:51617->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 172u IPv6 1407775 TCP luna.corn.cnwk:48692->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 173u IPv6 1407776 TCP luna.corn.cnwk:42208->c17-dw-dev50-hdfs-dn-n1.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 174u IPv6 1407777 TCP luna.corn.cnwk:54258->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 175u IPv6 1407778 TCP luna.corn.cnwk:51621->c17-dw-dev50-hdfs-dn-n4.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 176u IPv6 1407779 TCP luna.corn.cnwk:54260->c17-dw-dev50-hdfs-dn-n2.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 177u IPv6 1407780 TCP luna.corn.cnwk:48697->c17-dw-dev50-hdfs-dn-n3.corn.com:50010 (CLOSE_WAIT) java 19625 mdesnoyer 178r FIFO 0,6 1407966 pipe java 19625 mdesnoyer 179w FIFO 0,6 1407966 pipe java 19625 mdesnoyer 180u 0000 0,7 0 32 anon_inode On Mon, Aug 3, 2009 at 6:10 PM, Grant Ingersoll <[email protected]> wrote: > Attachments usually don't come through on ASF mailing lists. > > Also, Mahout trunk requires Hadoop 0.20.0 > > > > On Aug 3, 2009, at 6:05 PM, Mark Desnoyer wrote: > > Hi, >> >> Has anybody had a problem running KMeans leaking TCP connections, which >> eventually causes the OS to issue a "Too Many Files Open" error? In other >> words, it looks like a file descriptor leak. >> >> I'm running a four machine cluster of Hadoop 18.3, with the current >> version of Mahout from the trunk. Each k-means iteration is causing an extra >> 40 or so TCP connections to be created that get orphaned in CLOSE_WAIT. I've >> attached the output of lsof from the middle of a run. >> >> Any help would be appreciated! >> >> Cheers, >> Mark >> > > >
