Well, the problem is that there is a thread leak. It is not clear whether the file creation succeeded.
It isn't that surprising that there is a thread leak since Hadoop code is pretty cavalier about this and the Hadoop test cases don't check for this. I think that there might be a way to signal that this is ok, possibly with annotations. Somebody who knows the test randomizer framework would have to comment on that. On Mon, Jun 16, 2014 at 6:38 PM, Wei Zhang <[email protected]> wrote: > Thanks, Ted. I did try to do the fs.close(); The code looks like this: > @Test > *public* *void* simpleTest() *throws* IOException{ > Configuration conf = *new* Configuration(); > FileSystem fs = FileSystem.*get*(conf); > Path filenamePath = *new* Path("/tmp/tmp.txt"); > FSDataOutputStream out = fs.create(filenamePath); > out.close(); > fs.close(); > } > > It still generates the same error message. > > Thanks! > > Wei > > [image: Inactive hide details for Ted Dunning ---06/16/2014 09:29:28 > PM---Close the FileSystem object (fs). On Mon, Jun 16, 2014 at 6:2]Ted > Dunning ---06/16/2014 09:29:28 PM---Close the FileSystem object (fs). On > Mon, Jun 16, 2014 at 6:22 PM, Wei Zhang <[email protected]> wrote > > From: Ted Dunning <[email protected]> > To: "[email protected]" <[email protected]>, > Date: 06/16/2014 09:29 PM > Subject: Re: a seemingly benign test that fails MahoutTestCase > ------------------------------ > > > > Close the FileSystem object (fs). > > > > > On Mon, Jun 16, 2014 at 6:22 PM, Wei Zhang <[email protected]> wrote: > > > Hello Ted, > > > > Thank you very much for the reply! I have created a github gist that > > records the error message at > > https://gist.github.com/anonymous/21a2f36c995c1d39717b > > > > I am sorry that I forgot to attach the error log, it is something that > > looks like this (just in case that github gist is displaying > incorrectly): > > Jun 16, 2014 6:07:10 PM > > com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks > > WARNING: Will linger awaiting termination of 2 leaked thread(s). > > Jun 16, 2014 6:07:30 PM > > com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks > > SEVERE: 1 thread leaked from SUITE scope at > > org.apache.mahout.wei.inc.util.HDFSUtilTest: > > 1) Thread[id=15, name=org.apache.hadoop.hdfs.LeaseRenewer$1@7b968017, > > state=TIMED_WAITING, group=TGRP-HDFSUtilTest] > > at java.lang.Thread.sleep(*Native Method*) > > at > org.apache.hadoop.hdfs.LeaseRenewer.run(*LeaseRenewer.java:397* > > ) > > at org.apache.hadoop.hdfs.LeaseRenewer.access$600( > > *LeaseRenewer.java:69*) > > at org.apache.hadoop.hdfs.LeaseRenewer$1.run( > > *LeaseRenewer.java:273*) > > at java.lang.Thread.run(Unknown Source) > > Jun 16, 2014 6:07:30 PM > > com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll > > INFO: Starting to interrupt leaked threads: > > 1) Thread[id=15, name=org.apache.hadoop.hdfs.LeaseRenewer$1@7b968017, > > state=TIMED_WAITING, group=TGRP-HDFSUtilTest] > > Jun 16, 2014 6:07:30 PM > > com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll > > INFO: All leaked threads terminated. > > > > Thanks! > > > > Wei > > > > > > [image: Inactive hide details for Ted Dunning ---06/16/2014 07:50:30 > > PM---Attachments are stripped by the mailing list. Can you throw]Ted > > Dunning ---06/16/2014 07:50:30 PM---Attachments are stripped by the > mailing > > list. Can you throw the stack trace onto a github gist or s > > > > From: Ted Dunning <[email protected]> > > To: "[email protected]" <[email protected]>, > > Date: 06/16/2014 07:50 PM > > Subject: Re: a seemingly benign test that fails MahoutTestCase > > ------------------------------ > > > > > > > > Attachments are stripped by the mailing list. Can you throw the stack > > trace onto a github gist or something? > > > > > > On Mon, Jun 16, 2014 at 3:27 PM, Wei Zhang <[email protected]> wrote: > > > > > > > > > > > Hello, > > > > > > I am wrting a simple Unit test, which extends MahoutTestCase. I am > using > > > Mahout 0.9 and Hadoop 1.0.3. > > > > > > The test is quite simple, it just creates an empty file on HDFS. The > > entire > > > coding is the following: > > > > > > @Test > > > public void simpleTest() throws IOException{ > > > Configuration conf = new Configuration(); > > > FileSystem fs = FileSystem.get(conf); > > > Path filenamePath = new Path("/tmp/tmp.txt"); > > > FSDataOutputStream out = fs.create(filenamePath); > > > out.close(); > > > } > > > > > > However, this will fail the MahoutTestCase. In particular, the moment I > > > added the line 4 (FSDataOutputStream out = fs.create(filenamePath);), > > this > > > test would fail. > > > I am wondering can anyone give me some pointers regarding why this > naive > > > program would fail ? > > > The error log is attached below. > > > > > > Thank you very much ! > > > > > > Wei > > > > > >
