Nice article about the problem (now solved in Java 8): http://kirkwylie.blogspot.de/2008/09/solaris-10-terrible-choice-for-java.html
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Uwe Schindler [mailto:[email protected]] > Sent: Sunday, August 30, 2015 1:18 AM > To: [email protected] > Subject: RE: [JENKINS] Lucene-Solr-5.x-Solaris (multiarch/jdk1.7.0) - Build # > 9 > - Still Failing! > > Hi, > > This is a problem of Java 7 on Solaris. It still uses fork to spawn processes > in > Java 7; Java 8 has fixed this. Unfortunately the whole fork stuff is badly > implemented in Solaris and really allocates the same size of memory again > and does not use it - due to the large heaps this needs a lot of memory. > Workaround is to allocate enough Swap (which is never used): > > https://developer.opencloud.com/forum/posts/list/620.page > > For now I raised swap space (which is really simple to do with ZFS... Way > cool): > > root@solaris-vm:~# zfs set volsize=6g rpool/swap > > Super cool. > > In Java 8, it uses the new posix_spawn launch mechanism... (on Linux vfork). > This was (by the way) the buggy code that was fixed in U40, which caused > the Turkish Locale to fail :-) > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: [email protected] > > > > -----Original Message----- > > From: Uwe Schindler [mailto:[email protected]] > > Sent: Saturday, August 29, 2015 11:52 PM > > To: [email protected] > > Subject: RE: [JENKINS] Lucene-Solr-5.x-Solaris (multiarch/jdk1.7.0) - Build > > # > 9 > > - Still Failing! > > > > I am still digging... On Solaris there seems to be a general forking problem > on > > 32 bit processes. > > > > Uwe > > > > ----- > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: [email protected] > > > > > -----Original Message----- > > > From: Policeman Jenkins Server [mailto:[email protected]] > > > Sent: Saturday, August 29, 2015 11:02 PM > > > To: [email protected]; [email protected]; > [email protected] > > > Subject: [JENKINS] Lucene-Solr-5.x-Solaris (multiarch/jdk1.7.0) - Build # > > > 9 > - > > > Still Failing! > > > > > > Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Solaris/9/ > > > Java: multiarch/jdk1.7.0 -d32 -server -XX:+UseConcMarkSweepGC > > > > > > 4 tests failed. > > > FAILED: > > > > junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsNNFailoverTest > > > > > > Error Message: > > > Error while running command to get file permissions : > java.io.IOException: > > > Cannot run program "/bin/ls": error=12, Not enough space at > > > java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) at > > > org.apache.hadoop.util.Shell.runCommand(Shell.java:485) at > > > org.apache.hadoop.util.Shell.run(Shell.java:455) at > > > > > > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715 > > > ) at org.apache.hadoop.util.Shell.execCommand(Shell.java:808) at > > > org.apache.hadoop.util.Shell.execCommand(Shell.java:791) at > > > org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097) at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.lo > > > adPermissionInfo(RawLocalFileSystem.java:582) at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.g > > > etPermission(RawLocalFileSystem.java:557) at > > > > > > org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck( > > > DiskChecker.java:139) at > > > org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156) at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker. > > > checkDir(DataNode.java:2239) at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations( > > > DataNode.java:2281) at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNo > > > de.java:2263) at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(D > > > ataNode.java:2155) at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.jav > > > a:1443) at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:828) at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > at > > > > > > org.apache.solr.cloud.hdfs.HdfsNNFailoverTest.setupClass(HdfsNNFailoverT > > > est.java:44) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > > Method) at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > > > ava:57) at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > > > sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize > > > dRunner.java:1627) at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando > > > mizedRunner.java:776) at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando > > > mizedRunner.java:792) at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) at > > > > > > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1. > > > evaluate(SystemPropertiesRestoreRule.java:57) at > > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA > > > fterRule.java:46) at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) at > > > > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl > > > assName.java:42) at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) at > > > > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss > > > ertionsRequired.java:54) at > > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure > > > .java:48) at > > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule > > > IgnoreAfterMaxFailures.java:65) at > > > > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore > > > TestSuites.java:55) at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) at > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > > > run(ThreadLeakControl.java:365) at > java.lang.Thread.run(Thread.java:745) > > > Caused by: java.io.IOException: error=12, Not enough space at > > > java.lang.UNIXProcess.forkAndExec(Native Method) at > > > java.lang.UNIXProcess.<init>(UNIXProcess.java:137) at > > > java.lang.ProcessImpl.start(ProcessImpl.java:130) at > > > java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) ... 44 more > > > > > > Stack Trace: > > > java.lang.RuntimeException: Error while running command to get file > > > permissions : java.io.IOException: Cannot run program "/bin/ls": > error=12, > > > Not enough space > > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) > > > at org.apache.hadoop.util.Shell.runCommand(Shell.java:485) > > > at org.apache.hadoop.util.Shell.run(Shell.java:455) > > > at > > > > > > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715 > > > ) > > > at org.apache.hadoop.util.Shell.execCommand(Shell.java:808) > > > at org.apache.hadoop.util.Shell.execCommand(Shell.java:791) > > > at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097) > > > at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.lo > > > adPermissionInfo(RawLocalFileSystem.java:582) > > > at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.g > > > etPermission(RawLocalFileSystem.java:557) > > > at > > > > > > org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck( > > > DiskChecker.java:139) > > > at > > > org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156) > > > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker. > > > checkDir(DataNode.java:2239) > > > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations( > > > DataNode.java:2281) > > > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNo > > > de.java:2263) > > > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(D > > > ataNode.java:2155) > > > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.jav > > > a:1443) > > > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:828) > > > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > > at > > > > > > org.apache.solr.cloud.hdfs.HdfsNNFailoverTest.setupClass(HdfsNNFailoverT > > > est.java:44) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > > > ava:57) > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > > > sorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize > > > dRunner.java:1627) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando > > > mizedRunner.java:776) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando > > > mizedRunner.java:792) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1. > > > evaluate(SystemPropertiesRestoreRule.java:57) > > > at > > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA > > > fterRule.java:46) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl > > > assName.java:42) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss > > > ertionsRequired.java:54) > > > at > > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure > > > .java:48) > > > at > > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule > > > IgnoreAfterMaxFailures.java:65) > > > at > > > > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore > > > TestSuites.java:55) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > > > run(ThreadLeakControl.java:365) > > > at java.lang.Thread.run(Thread.java:745) > > > Caused by: java.io.IOException: error=12, Not enough space > > > at java.lang.UNIXProcess.forkAndExec(Native Method) > > > at java.lang.UNIXProcess.<init>(UNIXProcess.java:137) > > > at java.lang.ProcessImpl.start(ProcessImpl.java:130) > > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) > > > ... 44 more > > > > > > at __randomizedtesting.SeedInfo.seed([5D8F351977870E3F]:0) > > > at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.lo > > > adPermissionInfo(RawLocalFileSystem.java:620) > > > at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.g > > > etPermission(RawLocalFileSystem.java:557) > > > at > > > > > > org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck( > > > DiskChecker.java:139) > > > at > > > org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156) > > > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker. > > > checkDir(DataNode.java:2239) > > > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations( > > > DataNode.java:2281) > > > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNo > > > de.java:2263) > > > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(D > > > ataNode.java:2155) > > > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.jav > > > a:1443) > > > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:828) > > > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > > at > > > > > > org.apache.solr.cloud.hdfs.HdfsNNFailoverTest.setupClass(HdfsNNFailoverT > > > est.java:44) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > > > ava:57) > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > > > sorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize > > > dRunner.java:1627) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando > > > mizedRunner.java:776) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando > > > mizedRunner.java:792) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1. > > > evaluate(SystemPropertiesRestoreRule.java:57) > > > at > > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA > > > fterRule.java:46) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl > > > assName.java:42) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss > > > ertionsRequired.java:54) > > > at > > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure > > > .java:48) > > > at > > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule > > > IgnoreAfterMaxFailures.java:65) > > > at > > > > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore > > > TestSuites.java:55) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > > > run(ThreadLeakControl.java:365) > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > FAILED: > > > junit.framework.TestSuite.org.apache.solr.store.hdfs.HdfsDirectoryTest > > > > > > Error Message: > > > access denied ("java.io.FilePermission" > > > "/export/home/jenkins/workspace/Lucene-Solr-5.x- > Solaris/solr/build/solr- > > > core/test/J1" "write") > > > > > > Stack Trace: > > > java.security.AccessControlException: access denied > > > ("java.io.FilePermission" "/export/home/jenkins/workspace/Lucene- > Solr- > > > 5.x-Solaris/solr/build/solr-core/test/J1" "write") > > > at __randomizedtesting.SeedInfo.seed([5D8F351977870E3F]:0) > > > at > > > > > > java.security.AccessControlContext.checkPermission(AccessControlContext.j > > > ava:395) > > > at > > > > java.security.AccessController.checkPermission(AccessController.java:559) > > > at > > > java.lang.SecurityManager.checkPermission(SecurityManager.java:549) > > > at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) > > > at java.io.File.canWrite(File.java:785) > > > at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:1002) > > > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.createPermissionsDiagnosisString( > > > MiniDFSCluster.java:856) > > > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:812) > > > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:60) > > > at > > > > > > org.apache.solr.store.hdfs.HdfsDirectoryTest.beforeClass(HdfsDirectoryTest. > > > java:62) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > > > ava:57) > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > > > sorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize > > > dRunner.java:1627) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando > > > mizedRunner.java:776) > > > at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando > > > mizedRunner.java:792) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1. > > > evaluate(SystemPropertiesRestoreRule.java:57) > > > at > > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA > > > fterRule.java:46) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl > > > assName.java:42) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss > > > ertionsRequired.java:54) > > > at > > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure > > > .java:48) > > > at > > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule > > > IgnoreAfterMaxFailures.java:65) > > > at > > > > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore > > > TestSuites.java:55) > > > at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > > > run(ThreadLeakControl.java:365) > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > FAILED: > > > junit.framework.TestSuite.org.apache.solr.store.hdfs.HdfsDirectoryTest > > > > > > Error Message: > > > 1 thread leaked from SUITE scope at > > > org.apache.solr.store.hdfs.HdfsDirectoryTest: 1) Thread[id=20389, > > > name=IPC Server idle connection scanner for port 41610, state=WAITING, > > > group=TGRP-HdfsDirectoryTest] at java.lang.Object.wait(Native > > > Method) at java.lang.Object.wait(Object.java:503) at > > > java.util.TimerThread.mainLoop(Timer.java:526) at > > > java.util.TimerThread.run(Timer.java:505) > > > > > > Stack Trace: > > > com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked > > from > > > SUITE scope at org.apache.solr.store.hdfs.HdfsDirectoryTest: > > > 1) Thread[id=20389, name=IPC Server idle connection scanner for port > > > 41610, state=WAITING, group=TGRP-HdfsDirectoryTest] > > > at java.lang.Object.wait(Native Method) > > > at java.lang.Object.wait(Object.java:503) > > > at java.util.TimerThread.mainLoop(Timer.java:526) > > > at java.util.TimerThread.run(Timer.java:505) > > > at __randomizedtesting.SeedInfo.seed([5D8F351977870E3F]:0) > > > > > > > > > FAILED: > > > junit.framework.TestSuite.org.apache.solr.store.hdfs.HdfsDirectoryTest > > > > > > Error Message: > > > There are still zombie threads that couldn't be terminated: 1) > > > Thread[id=20389, name=IPC Server idle connection scanner for port > 41610, > > > state=WAITING, group=TGRP-HdfsDirectoryTest] at > > > java.lang.Object.wait(Native Method) at > > > java.lang.Object.wait(Object.java:503) at > > > java.util.TimerThread.mainLoop(Timer.java:526) at > > > java.util.TimerThread.run(Timer.java:505) > > > > > > Stack Trace: > > > com.carrotsearch.randomizedtesting.ThreadLeakError: There are still > > zombie > > > threads that couldn't be terminated: > > > 1) Thread[id=20389, name=IPC Server idle connection scanner for port > > > 41610, state=WAITING, group=TGRP-HdfsDirectoryTest] > > > at java.lang.Object.wait(Native Method) > > > at java.lang.Object.wait(Object.java:503) > > > at java.util.TimerThread.mainLoop(Timer.java:526) > > > at java.util.TimerThread.run(Timer.java:505) > > > at __randomizedtesting.SeedInfo.seed([5D8F351977870E3F]:0) > > > > > > > > > > > > > > > Build Log: > > > [...truncated 10577 lines...] > > > [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsNNFailoverTest > > > [junit4] 2> Creating dataDir: > /export/home/jenkins/workspace/Lucene- > > > Solr-5.x-Solaris/solr/build/solr- > > > > > > core/test/J0/temp/solr.cloud.hdfs.HdfsNNFailoverTest_5D8F351977870E3F- > > > 001/init-core-data-001 > > > [junit4] 2> 2599844 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] > > o.a.s.BaseDistributedSearchTestCase > > > Setting hostContext system property: / > > > [junit4] 2> 2616331 WARN (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.h.u.NativeCodeLoader > Unable > > to > > > load native-hadoop library for your platform... using builtin-java classes > > > where applicable > > > [junit4] 1> Formatting using clusterid: testClusterID > > > [junit4] 2> 2617524 WARN (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot > > > locate configuration: tried hadoop-metrics2- > > namenode.properties,hadoop- > > > metrics2.properties > > > [junit4] 2> 2617755 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Logging to > > > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > > > org.mortbay.log.Slf4jLog > > > [junit4] 2> 2617771 WARN (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty > > > request log can only be enabled using Log4j > > > [junit4] 2> 2617878 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log jetty-6.1.26 > > > [junit4] 2> 2617942 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Extract > > > jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop- > > > hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to > > > ./temp/Jetty_solaris.vm_35231_hdfs____thayv4/webapp > > > [junit4] 2> 2618129 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log NO JSP Support for /, did > > not > > > find org.apache.jasper.servlet.JspServlet > > > [junit4] 2> 2619464 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Started > > > HttpServer2$SelectChannelConnectorWithSafeStartup@solaris-vm:35231 > > > [junit4] 2> 2637264 WARN (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.h.h.s.d.DataNode Invalid > > > dfs.datanode.data.dir /export/home/jenkins/workspace/Lucene-Solr- > 5.x- > > > Solaris/solr/build/solr- > > > > > > core/test/J0/temp/solr.cloud.hdfs.HdfsNNFailoverTest_5D8F351977870E3F- > > > 001/tempDir-001/hdfsBaseDir/data/data2 : > > > [junit4] 2> java.io.IOException: Cannot run program "chmod": > error=12, > > > Not enough space > > > [junit4] 2> at > java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) > > > [junit4] 2> at > org.apache.hadoop.util.Shell.runCommand(Shell.java:485) > > > [junit4] 2> at org.apache.hadoop.util.Shell.run(Shell.java:455) > > > [junit4] 2> at > > > > > > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715 > > > ) > > > [junit4] 2> at > > > org.apache.hadoop.util.Shell.execCommand(Shell.java:808) > > > [junit4] 2> at > > > org.apache.hadoop.util.Shell.execCommand(Shell.java:791) > > > [junit4] 2> at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSyste > > > m.java:656) > > > [junit4] 2> at > > > > > > org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:4 > > > 90) > > > [junit4] 2> at > > > > > > org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck( > > > DiskChecker.java:140) > > > [junit4] 2> at > > > org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker. > > > checkDir(DataNode.java:2239) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations( > > > DataNode.java:2281) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNo > > > de.java:2263) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(D > > > ataNode.java:2155) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.jav > > > a:1443) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:828) > > > [junit4] 2> at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > > [junit4] 2> at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > > [junit4] 2> at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > > [junit4] 2> at > > > > > > org.apache.solr.cloud.hdfs.HdfsNNFailoverTest.setupClass(HdfsNNFailoverT > > > est.java:44) > > > [junit4] 2> at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > > Method) > > > [junit4] 2> at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > > > ava:57) > > > [junit4] 2> at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > > > sorImpl.java:43) > > > [junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize > > > dRunner.java:1627) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando > > > mizedRunner.java:776) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando > > > mizedRunner.java:792) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1. > > > evaluate(SystemPropertiesRestoreRule.java:57) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA > > > fterRule.java:46) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl > > > assName.java:42) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss > > > ertionsRequired.java:54) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure > > > .java:48) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule > > > IgnoreAfterMaxFailures.java:65) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore > > > TestSuites.java:55) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > > > run(ThreadLeakControl.java:365) > > > [junit4] 2> at java.lang.Thread.run(Thread.java:745) > > > [junit4] 2> Caused by: java.io.IOException: error=12, Not enough > > > space > > > [junit4] 2> at java.lang.UNIXProcess.forkAndExec(Native > Method) > > > [junit4] 2> at > java.lang.UNIXProcess.<init>(UNIXProcess.java:137) > > > [junit4] 2> at java.lang.ProcessImpl.start(ProcessImpl.java:130) > > > [junit4] 2> at > java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) > > > [junit4] 2> ... 43 more > > > [junit4] 2> 2637287 WARN > > > (org.apache.hadoop.util.JvmPauseMonitor$Monitor@be51b7) [ ] > > > o.a.h.u.JvmPauseMonitor Detected pause in JVM or host machine (eg > GC): > > > pause of approximately 15969ms > > > [junit4] 2> No GCs detected > > > [junit4] 2> 2637368 WARN (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty > > > request log can only be enabled using Log4j > > > [junit4] 2> 2637384 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log jetty-6.1.26 > > > [junit4] 2> 2637422 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Extract > > > jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop- > > > hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to > > > ./temp/Jetty_solaris.vm_49465_datanode____96t731/webapp > > > [junit4] 2> 2637655 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log NO JSP Support for /, did > > not > > > find org.apache.jasper.servlet.JspServlet > > > [junit4] 2> 2638756 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Started > > > HttpServer2$SelectChannelConnectorWithSafeStartup@solaris-vm:49465 > > > [junit4] 2> 2645079 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Stopped > > > HttpServer2$SelectChannelConnectorWithSafeStartup@solaris-vm:0 > > > [junit4] 2> 2645234 ERROR (DataNode: > > > [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-5.x- > > > Solaris/solr/build/solr- > > > > > > core/test/J0/temp/solr.cloud.hdfs.HdfsNNFailoverTest_5D8F351977870E3F- > > > 001/tempDir-001/hdfsBaseDir/data/data1/, > > > [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-5.x- > > > Solaris/solr/build/solr- > > > > > > core/test/J0/temp/solr.cloud.hdfs.HdfsNNFailoverTest_5D8F351977870E3F- > > > 001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to solaris- > > > vm/127.0.0.1:61051) [ ] o.a.h.h.s.d.DataNode Initialization failed for > Block > > > pool <registering> (Datanode Uuid unassigned) service to solaris- > > > vm/127.0.0.1:61051. Exiting. > > > [junit4] 2> java.io.IOException: DN shut down before block pool > > > connected > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.datanode.BPServiceActor.retrieveNamespac > > > eInfo(BPServiceActor.java:185) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAnd > > > Handshake(BPServiceActor.java:215) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceAct > > > or.java:828) > > > [junit4] 2> at java.lang.Thread.run(Thread.java:745) > > > [junit4] 2> 2645236 WARN (DataNode: > > > [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-5.x- > > > Solaris/solr/build/solr- > > > > > > core/test/J0/temp/solr.cloud.hdfs.HdfsNNFailoverTest_5D8F351977870E3F- > > > 001/tempDir-001/hdfsBaseDir/data/data1/, > > > [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-5.x- > > > Solaris/solr/build/solr- > > > > > > core/test/J0/temp/solr.cloud.hdfs.HdfsNNFailoverTest_5D8F351977870E3F- > > > 001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to solaris- > > > vm/127.0.0.1:61051) [ ] o.a.h.h.s.d.DataNode Ending block pool service > > for: > > > Block pool <registering> (Datanode Uuid unassigned) service to solaris- > > > vm/127.0.0.1:61051 > > > [junit4] 2> 2645259 WARN > > > > > > (org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager > > > $Monitor@7b7964) [ ] o.a.h.h.s.b.DecommissionManager Monitor > > > interrupted: java.lang.InterruptedException: sleep interrupted > > > [junit4] 2> 2645314 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Stopped > > > HttpServer2$SelectChannelConnectorWithSafeStartup@solaris-vm:0 > > > [junit4] 2> 2645418 ERROR (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.h.m.l.MethodMetric Error > > > invoking method getBlocksTotal > > > [junit4] 2> java.lang.reflect.InvocationTargetException > > > [junit4] 2> at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > > Method) > > > [junit4] 2> at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > > > ava:57) > > > [junit4] 2> at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > > > sorImpl.java:43) > > > [junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.j > > > ava:111) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.jav > > > a:144) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.ja > > > va:387) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsS > > > ourceBuilder.java:79) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(Metrics > > > SourceAdapter.java:195) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache( > > > MetricsSourceAdapter.java:172) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(Me > > > tricsSourceAdapter.java:151) > > > [junit4] 2> at > > > > > > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(De > > > faultMBeanServerInterceptor.java:1804) > > > [junit4] 2> at > > > > > > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassNam > > > e(DefaultMBeanServerInterceptor.java:1595) > > > [junit4] 2> at > > > > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPer > > > mission(DefaultMBeanServerInterceptor.java:1813) > > > [junit4] 2> at > > > > > > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregist > > > erMBean(DefaultMBeanServerInterceptor.java:430) > > > [junit4] 2> at > > > > > > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean > > > (DefaultMBeanServerInterceptor.java:415) > > > [junit4] 2> at > > > > > > com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanS > > > erver.java:546) > > > [junit4] 2> at > > > org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(Metri > > > csSourceAdapter.java:227) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourc > > > eAdapter.java:212) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsS > > > ystemImpl.java:461) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemIm > > > pl.java:212) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSyst > > > emImpl.java:592) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(D > > > efaultMetricsSystem.java:72) > > > [junit4] 2> at > > > > > > org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMe > > > tricsSystem.java:68) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shut > > > down(NameNodeMetrics.java:145) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.jav > > > a:822) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:172 > > > 0) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:169 > > > 9) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:838) > > > [junit4] 2> at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > > [junit4] 2> at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > > [junit4] 2> at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > > [junit4] 2> at > > > > > > org.apache.solr.cloud.hdfs.HdfsNNFailoverTest.setupClass(HdfsNNFailoverT > > > est.java:44) > > > [junit4] 2> at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > > Method) > > > [junit4] 2> at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > > > ava:57) > > > [junit4] 2> at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > > > sorImpl.java:43) > > > [junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize > > > dRunner.java:1627) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando > > > mizedRunner.java:776) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando > > > mizedRunner.java:792) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1. > > > evaluate(SystemPropertiesRestoreRule.java:57) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA > > > fterRule.java:46) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl > > > assName.java:42) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > > > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss > > > ertionsRequired.java:54) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure > > > .java:48) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule > > > IgnoreAfterMaxFailures.java:65) > > > [junit4] 2> at > > > > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore > > > TestSuites.java:55) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat > > > ementAdapter.java:36) > > > [junit4] 2> at > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > > > run(ThreadLeakControl.java:365) > > > [junit4] 2> at java.lang.Thread.run(Thread.java:745) > > > [junit4] 2> Caused by: java.lang.NullPointerException > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksM > > > ap.java:198) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBl > > > ocks(BlockManager.java:3291) > > > [junit4] 2> at > > > > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(F > > > SNamesystem.java:6223) > > > [junit4] 2> ... 58 more > > > [junit4] 2> 2645432 INFO (SUITE-HdfsNNFailoverTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.s.SolrTestCaseJ4 > > ###deleteCore > > > [junit4] 2> NOTE: test params are: codec=Asserting(Lucene53), > > > sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, > > > locale=mk_MK, timezone=Asia/Shanghai > > > [junit4] 2> NOTE: SunOS 5.11 x86/Oracle Corporation 1.7.0_85 (32- > > > bit)/cpus=3,threads=1,free=99794816,total=518979584 > > > [junit4] 2> NOTE: All tests run in this JVM: [SolrCloudExampleTest, > > > TestStressVersions, TestSerializedLuceneMatchVersion, TestSolrJ, > > > DistanceUnitsTest, MultiThreadedOCPTest, TestDistribDocBasedVersion, > > > BJQParserTest, ZkCLITest, QueryEqualityTest, PrimitiveFieldTypeTest, > > > DistributedQueryComponentOptimizationTest, AliasIntegrationTest, > > > TestInitQParser, TestAuthorizationFramework, TestLazyCores, > > > SolrIndexConfigTest, TestFunctionQuery, TestXIncludeConfig, > > > HardAutoCommitTest, DocValuesMultiTest, TestDefaultStatsCache, > > > SolrRequestParserTest, RecoveryZkTest, UpdateParamsTest, > > > TestSolrDeletionPolicy1, TestDFRSimilarityFactory, TestFastWriter, > > > PathHierarchyTokenizerFactoryTest, TestDynamicLoading, > > > TestElisionMultitermQuery, PolyFieldTest, UnloadDistributedZkTest, > > > TestJsonRequest, TestRuleBasedAuthorizationPlugin, > > > TestManagedStopFilterFactory, TestRawResponseWriter, > > IndexSchemaTest, > > > TestEmbeddedSolrServerConstructors, InfoHandlerTest, > > > AlternateDirectoryTest, LeaderElectionTest, JsonLoaderTest, > > > TestCoreContainer, DirectSolrSpellCheckerTest, RequestLoggingTest, > > > ZkNodePropsTest, TermsComponentTest, TestConfig, > > > TestFieldTypeCollectionResource, XsltUpdateRequestHandlerTest, > > > TestManagedSchemaFieldResource, TestSchemaResource, > > > DataDrivenBlockJoinTest, TestExactStatsCache, TestConfigSetProperties, > > > DeleteLastCustomShardedReplicaTest, TestAnalyzedSuggestions, > > > DirectUpdateHandlerTest, ExternalFileFieldSortTest, > > TestIBSimilarityFactory, > > > TestMissingGroups, ClusterStateUpdateTest, ActionThrottleTest, > > > QueryElevationComponentTest, DocValuesTest, QueryResultKeyTest, > > > TestLRUCache, TestPhraseSuggestions, SimplePostToolTest, > > > TriLevelCompositeIdRoutingTest, DistributedMLTComponentTest, > > > CloudExitableDirectoryReaderTest, TestSolrCloudWithKerberosAlt, > > > TestCodecSupport, TestConfigSets, PeerSyncTest, > > > XmlUpdateRequestHandlerTest, SpatialHeatmapFacetsTest, > > > SoftAutoCommitTest, TestSchemaNameResource, > > > PreAnalyzedUpdateProcessorTest, TestJmxMonitoredMap, > > > TestDistributedStatsComponentCardinality, > > > TestManagedSynonymFilterFactory, JSONWriterTest, TestNRTOpen, > > > ReplicationFactorTest, DOMUtilTest, SolrCoreTest, > > > DocExpirationUpdateProcessorFactoryTest, FastVectorHighlighterTest, > > > SuggesterFSTTest, TestExtendedDismaxParser, TestSolrConfigHandler, > > > DocumentAnalysisRequestHandlerTest, > > > DistributedFacetPivotSmallAdvancedTest, BlockDirectoryTest, > > > TestQuerySenderNoQuery, TestHashPartitioner, DateFieldTest, > > > SegmentsInfoRequestHandlerTest, TestFieldCollectionResource, > > > RecoveryAfterSoftCommitTest, TestMergePolicyConfig, > > TestFieldSortValues, > > > SecurityConfHandlerTest, TestStressReorder, BufferStoreTest, > > > TestRandomRequestDistribution, HdfsBasicDistributedZkTest, > > > TestCloudManagedSchemaConcurrent, TestReplicaProperties, > > > DisMaxRequestHandlerTest, TestMacros, TestStressLucene, > > > TestReloadAndDeleteDocs, BasicAuthIntegrationTest, TestDocSet, > > > BasicDistributedZkTest, DistributedQueryElevationComponentTest, > > > TestGroupingSearch, TestObjectReleaseTracker, > MoreLikeThisHandlerTest, > > > OverseerTest, TestFaceting, TestUpdate, TestClassNameShortening, > > > TestRestManager, SyncSliceTest, ShardRoutingTest, ZkSolrClientTest, > > > TestZkChroot, TestRandomDVFaceting, ShardRoutingCustomTest, > > > TestDistributedGrouping, DistributedSpellCheckComponentTest, > > > ZkControllerTest, TestRealTimeGet, TestReload, > > > DistributedTermsComponentTest, TestRangeQuery, SimpleFacetsTest, > > > TestSolr4Spatial, StatsComponentTest, SolrCmdDistributorTest, TestSort, > > > CurrencyFieldXmlFileTest, AnalysisAfterCoreReloadTest, > > > TestFoldingMultitermQuery, SuggesterTSTTest, TestCSVLoader, > > > SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, > > > DirectUpdateHandlerOptimizeTest, > > > StatelessScriptUpdateProcessorFactoryTest, DistanceFunctionTest, > > > IndexBasedSpellCheckerTest, StandardRequestHandlerTest, > > > TestOmitPositions, DocumentBuilderTest, RequiredFieldsTest, > > > TestArbitraryIndexDir, LoggingHandlerTest, ReturnFieldsTest, > > > MBeansHandlerTest, UniqFieldsUpdateProcessorFactoryTest, > > > PingRequestHandlerTest, TestComponentsName, TestLFUCache, > > > PreAnalyzedFieldTest, TestSystemIdResolver, > SpellingQueryConverterTest, > > > TestUtils, TestDocumentBuilder, SliceStateTest, SystemInfoHandlerTest, > > > UUIDFieldTest, FileUtilsTest, CircularListTest, TestRTGBase, > > > CursorPagingTest, DistributedIntervalFacetingTest, > > > TestDistributedMissingSort, TestSimpleTrackingShardHandler, > > > AsyncMigrateRouteKeyTest, DeleteInactiveReplicaTest, > > > DistribDocExpirationUpdateProcessorTest, > > LeaderFailoverAfterPartitionTest, > > > OverriddenZkACLAndCredentialsProvidersTest, > > > OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, > > > OverseerTaskQueueTest, SSLMigrationTest, SaslZkACLProviderTest, > > > SimpleCollectionCreateDeleteTest, TestAuthenticationFramework, > > > TestCloudInspectUtil, TestCollectionAPI, TestMiniSolrCloudClusterSSL, > > > TestRebalanceLeaders, TestRequestStatusCollectionAPI, > > > HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, > > > HdfsCollectionsAPIDistributedZkTest, HdfsNNFailoverTest] > > > [junit4] 2> NOTE: reproduce with: ant test - > > > Dtestcase=HdfsNNFailoverTest -Dtests.seed=5D8F351977870E3F - > > > Dtests.slow=true -Dtests.locale=mk_MK - > Dtests.timezone=Asia/Shanghai - > > > Dtests.asserts=true -Dtests.file.encoding=UTF-8 > > > [junit4] ERROR 0.00s J0 | HdfsNNFailoverTest (suite) <<< > > > [junit4] > Throwable #1: java.lang.RuntimeException: Error while > > running > > > command to get file permissions : java.io.IOException: Cannot run > program > > > "/bin/ls": error=12, Not enough space > > > [junit4] > at > java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) > > > [junit4] > at > org.apache.hadoop.util.Shell.runCommand(Shell.java:485) > > > [junit4] > at org.apache.hadoop.util.Shell.run(Shell.java:455) > > > [junit4] > at > > > > > > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715 > > > ) > > > [junit4] > at > > > org.apache.hadoop.util.Shell.execCommand(Shell.java:808) > > > [junit4] > at > > > org.apache.hadoop.util.Shell.execCommand(Shell.java:791) > > > [junit4] > at > > > org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097) > > > [junit4] > at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.lo > > > adPermissionInfo(RawLocalFileSystem.java:582) > > > [junit4] > at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.g > > > etPermission(RawLocalFileSystem.java:557) > > > [junit4] > at > > > > > > org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck( > > > DiskChecker.java:139) > > > [junit4] > at > > > org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker. > > > checkDir(DataNode.java:2239) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations( > > > DataNode.java:2281) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNo > > > de.java:2263) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(D > > > ataNode.java:2155) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.jav > > > a:1443) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:828) > > > [junit4] > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > > [junit4] > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > > [junit4] > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > > [junit4] > at > > > > > > org.apache.solr.cloud.hdfs.HdfsNNFailoverTest.setupClass(HdfsNNFailoverT > > > est.java:44) > > > [junit4] > at java.lang.Thread.run(Thread.java:745) > > > [junit4] > Caused by: java.io.IOException: error=12, Not enough > > > space > > > [junit4] > at java.lang.UNIXProcess.forkAndExec(Native > Method) > > > [junit4] > at > java.lang.UNIXProcess.<init>(UNIXProcess.java:137) > > > [junit4] > at java.lang.ProcessImpl.start(ProcessImpl.java:130) > > > [junit4] > at > java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) > > > [junit4] > ... 44 more > > > [junit4] > at > > > __randomizedtesting.SeedInfo.seed([5D8F351977870E3F]:0) > > > [junit4] > at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.lo > > > adPermissionInfo(RawLocalFileSystem.java:620) > > > [junit4] > at > > > > > > org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.g > > > etPermission(RawLocalFileSystem.java:557) > > > [junit4] > at > > > > > > org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck( > > > DiskChecker.java:139) > > > [junit4] > at > > > org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker. > > > checkDir(DataNode.java:2239) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations( > > > DataNode.java:2281) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNo > > > de.java:2263) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(D > > > ataNode.java:2155) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.jav > > > a:1443) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:828) > > > [junit4] > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > > [junit4] > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > > [junit4] > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > > [junit4] > at > > > > > > org.apache.solr.cloud.hdfs.HdfsNNFailoverTest.setupClass(HdfsNNFailoverT > > > est.java:44) > > > [junit4] > at java.lang.Thread.run(Thread.java:745) > > > [junit4] Completed [426/536] on J0 in 45.66s, 0 tests, 1 error <<< > > FAILURES! > > > > > > [...truncated 300 lines...] > > > [junit4] Suite: org.apache.solr.store.hdfs.HdfsDirectoryTest > > > [junit4] 2> Creating dataDir: > /export/home/jenkins/workspace/Lucene- > > > Solr-5.x-Solaris/solr/build/solr- > > > > core/test/J1/temp/solr.store.hdfs.HdfsDirectoryTest_5D8F351977870E3F- > > > 001/init-core-data-001 > > > [junit4] 2> 3147821 INFO (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized > > ssl > > > (false) and clientAuth (false) > > > [junit4] 1> Formatting using clusterid: testClusterID > > > [junit4] 2> 3147964 WARN (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot > > > locate configuration: tried hadoop-metrics2- > > namenode.properties,hadoop- > > > metrics2.properties > > > [junit4] 2> 3147974 WARN (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty > > > request log can only be enabled using Log4j > > > [junit4] 2> 3147976 INFO (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log jetty-6.1.26 > > > [junit4] 2> 3147994 INFO (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Extract > > > jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop- > > > hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to > > > ./temp/Jetty_solaris.vm_46547_hdfs____.vwfmpk/webapp > > > [junit4] 2> 3148170 INFO (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log NO JSP Support for /, did > > not > > > find org.apache.jasper.servlet.JspServlet > > > [junit4] 2> 3148982 INFO (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Started > > > HttpServer2$SelectChannelConnectorWithSafeStartup@solaris-vm:46547 > > > [junit4] 2> 3157264 INFO (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.m.log Stopped > > > HttpServer2$SelectChannelConnectorWithSafeStartup@solaris-vm:0 > > > [junit4] 2> 3157403 INFO (SUITE-HdfsDirectoryTest- > > > seed#[5D8F351977870E3F]-worker) [ ] o.a.s.SolrTestCaseJ4 > > ###deleteCore > > > [junit4] 2> Aug 29, 2015 9:01:07 PM > > > com.carrotsearch.randomizedtesting.ThreadLeakControl > > checkThreadLeaks > > > [junit4] 2> WARNING: Will linger awaiting termination of 1 leaked > > > thread(s). > > > [junit4] 2> Aug 29, 2015 9:01:27 PM > > > com.carrotsearch.randomizedtesting.ThreadLeakControl > > checkThreadLeaks > > > [junit4] 2> SEVERE: 1 thread leaked from SUITE scope at > > > org.apache.solr.store.hdfs.HdfsDirectoryTest: > > > [junit4] 2> 1) Thread[id=20389, name=IPC Server idle connection > > scanner > > > for port 41610, state=WAITING, group=TGRP-HdfsDirectoryTest] > > > [junit4] 2> at java.lang.Object.wait(Native Method) > > > [junit4] 2> at java.lang.Object.wait(Object.java:503) > > > [junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:526) > > > [junit4] 2> at java.util.TimerThread.run(Timer.java:505) > > > [junit4] 2> Aug 29, 2015 9:01:27 PM > > > com.carrotsearch.randomizedtesting.ThreadLeakControl > tryToInterruptAll > > > [junit4] 2> INFO: Starting to interrupt leaked threads: > > > [junit4] 2> 1) Thread[id=20389, name=IPC Server idle connection > > scanner > > > for port 41610, state=WAITING, group=TGRP-HdfsDirectoryTest] > > > [junit4] 2> Aug 29, 2015 9:01:30 PM > > > com.carrotsearch.randomizedtesting.ThreadLeakControl > tryToInterruptAll > > > [junit4] 2> SEVERE: There are still zombie threads that couldn't be > > > terminated: > > > [junit4] 2> 1) Thread[id=20389, name=IPC Server idle connection > > scanner > > > for port 41610, state=WAITING, group=TGRP-HdfsDirectoryTest] > > > [junit4] 2> at java.lang.Object.wait(Native Method) > > > [junit4] 2> at java.lang.Object.wait(Object.java:503) > > > [junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:526) > > > [junit4] 2> at java.util.TimerThread.run(Timer.java:505) > > > [junit4] 2> NOTE: test params are: codec=Asserting(Lucene53): {}, > > > docValues:{}, sim=DefaultSimilarity, locale=es_BO, > > > timezone=Antarctica/South_Pole > > > [junit4] 2> NOTE: SunOS 5.11 x86/Oracle Corporation 1.7.0_85 (32- > > > bit)/cpus=3,threads=2,free=136627544,total=518979584 > > > [junit4] 2> NOTE: All tests run in this JVM: > > > [TestIndexingPerformance, > > > TestCSVResponseWriter, DistributedQueryComponentCustomSortTest, > > > DirectSolrConnectionTest, FullSolrCloudDistribCmdsTest, > > > TestShardHandlerFactory, CacheHeaderTest, BasicZkTest, TestTrie, > > > FieldAnalysisRequestHandlerTest, PKIAuthenticationIntegrationTest, > > > OpenCloseCoreStressTest, TestSuggestSpellingConverter, > StressHdfsTest, > > > CleanupOldIndexTest, DistributedExpandComponentTest, > > > TestHdfsUpdateLog, TestSolrXml, TestAddFieldRealTimeGet, > > TestJsonFacets, > > > DistributedSuggestComponentTest, > > > OutOfBoxZkACLAndCredentialsProvidersTest, > AnalyticsMergeStrategyTest, > > > HLLUtilTest, ResponseHeaderTest, SearchHandlerTest, > > > BinaryUpdateRequestHandlerTest, DistributedFacetPivotWhiteBoxTest, > > > ConnectionManagerTest, SpellCheckComponentTest, > > > TestScoreJoinQPNoScore, SolrTestCaseJ4Test, SolrIndexSplitterTest, > > > TestConfigSetsAPI, TestDefaultSearchFieldResource, TestCryptoKeys, > > > TestNonDefinedSimilarityFactory, TestCoreDiscovery, RollingRestartTest, > > > SolrInfoMBeanTest, CustomCollectionTest, DistributedVersionInfoTest, > > > ClusterStateTest, TestReversedWildcardFilterFactory, SolrXmlInZkTest, > > > DistributedFacetPivotLongTailTest, URLClassifyProcessorTest, > > > TestLMJelinekMercerSimilarityFactory, RequestHandlersTest, > > > RemoteQueryErrorTest, LeaderElectionIntegrationTest, > > > SharedFSAutoReplicaFailoverTest, TestBadConfig, > > > SignatureUpdateProcessorFactoryTest, > > TestCursorMarkWithoutUniqueKey, > > > TestCrossCoreJoin, SparseHLLTest, DistributedQueueTest, > > > BigEndianAscendingWordSerializerTest, TestBM25SimilarityFactory, > > > AutoCommitTest, DateMathParserTest, BasicFunctionalityTest, > > > SuggesterWFSTTest, TestCollapseQParserPlugin, TestManagedResource, > > > TestSha256AuthenticationProvider, CollectionTooManyReplicasTest, > > > BadCopyFieldTest, TestDownShardTolerantSearch, > CloudMLTQParserTest, > > > NotRequiredUniqueKeyTest, TestAnalyzeInfixSuggestions, > > > ExitableDirectoryReaderTest, TestScoreJoinQPScore, DeleteShardTest, > > > RankQueryTest, TestSchemaManager, > > UpdateRequestProcessorFactoryTest, > > > CursorMarkTest, DistributedDebugComponentTest, DeleteReplicaTest, > > > RAMDirectoryFactoryTest, ConcurrentDeleteAndCreateCollectionTest, > > > TestQueryTypes, OutputWriterTest, TestSchemaSimilarityResource, > > > HighlighterMaxOffsetTest, ResponseLogComponentTest, > > > TestCloudPivotFacet, DocValuesMissingTest, > > > FieldMutatingUpdateProcessorTest, HttpPartitionTest, TestCollationField, > > > ZkStateWriterTest, TestQuerySenderListener, AtomicUpdatesTest, > > > TestStressRecovery, TestRandomFaceting, > > > SharedFSAutoReplicaFailoverUtilsTest, CoreAdminHandlerTest, > > > HighlighterConfigTest, TestCustomSort, MultiTermTest, > > > VMParamsZkACLAndCredentialsProvidersTest, > > > IgnoreCommitOptimizeUpdateProcessorFactoryTest, > CollectionReloadTest, > > > PrimUtilsTest, TestRecovery, TestWriterPerf, > > > AddSchemaFieldsUpdateProcessorFactoryTest, TimeZoneUtilsTest, > > > CurrencyFieldOpenExchangeTest, TestSolrCLIRunExample, > > > TestPHPSerializedResponseWriter, ChaosMonkeySafeLeaderTest, > > > TestIndexSearcher, EnumFieldTest, TestSolrIndexConfig, > > > TermVectorComponentDistributedTest, TestJoin, > TestExpandComponent, > > > TestManagedResourceStorage, SortByFunctionTest, > > > TestDefaultSimilarityFactory, SuggesterTest, TestValueSourceCache, > > > SolrPluginUtilsTest, TermVectorComponentTest, TestFiltering, > > > TestQueryUtils, FileBasedSpellCheckerTest, BasicDistributedZk2Test, > > > CollectionsAPIDistributedZkTest, TestReplicationHandler, > > > TestDistributedSearch, BadIndexSchemaTest, ConvertedLegacyTest, > > > HighlighterTest, ShowFileRequestHandlerTest, SpellCheckCollatorTest, > > > SpatialFilterTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, > > > TestPseudoReturnFields, TestAtomicUpdateErrorCases, > > > TestWordDelimiterFilterFactory, DefaultValueUpdateProcessorTest, > > > TestRemoteStreaming, DebugComponentTest, > TestSurroundQueryParser, > > > LukeRequestHandlerTest, TestSolrQueryParser, > > > IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, > > > TestJmxIntegration, QueryParsingTest, TestPartialUpdateDeduplication, > > > CSVRequestHandlerTest, TestBinaryResponseWriter, SOLR749Test, > > > CopyFieldTest, BadComponentTest, TestSolrDeletionPolicy2, SampleTest, > > > TestBinaryField, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, > > > TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, > > > TestPostingsSolrHighlighter, TestLuceneMatchVersion, > > > SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, > > > EchoParamsTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, > > > TestLMDirichletSimilarityFactory, ResourceLoaderTest, > > > TestFastOutputStream, ScriptEngineTest, > > > OpenExchangeRatesOrgProviderTest, PluginInfoTest, TestFastLRUCache, > > > ChaosMonkeyNothingIsSafeTest, TestHighlightDedupGrouping, > > > TestTolerantSearch, TestJettySolrRunner, AssignTest, > > > AsyncCallRequestStatusResponseTest, CollectionStateFormat2Test, > > > CollectionsAPIAsyncDistributedZkTest, DistribCursorPagingTest, > > > DistribJoinFromCollectionTest, LeaderInitiatedRecoveryOnCommitTest, > > > MigrateRouteKeyTest, OverseerStatusTest, ShardSplitTest, > > > TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, > > > TestLeaderElectionZkExpiry, TestMiniSolrCloudCluster, > > > TestShortCircuitedRequests, HdfsRecoverLeaseTest, > > > CachingDirectoryFactoryTest, HdfsDirectoryFactoryTest, > TestConfigOverlay, > > > TestConfigSetImmutable, TestImplicitCoreProperties, > > > TestInfoStreamLogging, TestInitParams, TestSolrDynamicMBean, > > > TestBlobHandler, TestConfigReload, TestReplicationHandlerBackup, > > > TestSolrConfigHandlerConcurrent, CoreAdminCreateDiscoverTest, > > > CoreAdminRequestStatusTest, CoreMergeIndexesAdminHandlerTest, > > > DistributedFacetPivotLargeTest, DistributedFacetPivotSmallTest, > > > FacetPivotSmallTest, SuggestComponentTest, JavabinLoaderTest, > > > SmileWriterTest, TestIntervalFaceting, TestChildDocTransformer, > > > TestCustomDocTransformer, TestSortingResponseWriter, > > > TestBulkSchemaAPI, TestFieldResource, > > > TestManagedSchemaDynamicFieldResource, > TestBulkSchemaConcurrent, > > > TestCloudSchemaless, TestReloadDeadlock, TestSearcherReuse, > > > TestSimpleQParserPlugin, TestSmileRequest, TestSolr4Spatial2, > > > TestStandardQParsers, TestStressUserVersions, TestTrieFacet, > > > TestMinMaxOnMultiValuedField, TestOrdValues, > > > TestSortByMinMaxFunction, SimpleMLTQParserTest, TestDistribIDF, > > > TestExactSharedStatsCache, TestPKIAuthenticationPlugin, > > > TestBlendedInfixSuggestions, TestFileDictionaryLookup, > > > TestFreeTextSuggestions, TestHighFrequencyDictionaryFactory, > > > BlockCacheTest, HdfsDirectoryTest] > > > [junit4] 2> NOTE: reproduce with: ant test - > Dtestcase=HdfsDirectoryTest > > - > > > Dtests.seed=5D8F351977870E3F -Dtests.slow=true -Dtests.locale=es_BO > - > > > Dtests.timezone=Antarctica/South_Pole -Dtests.asserts=true - > > > Dtests.file.encoding=UTF-8 > > > [junit4] ERROR 0.00s J1 | HdfsDirectoryTest (suite) <<< > > > [junit4] > Throwable #1: java.security.AccessControlException: > > > access > > > denied ("java.io.FilePermission" > > "/export/home/jenkins/workspace/Lucene- > > > Solr-5.x-Solaris/solr/build/solr-core/test/J1" "write") > > > [junit4] > at > > > __randomizedtesting.SeedInfo.seed([5D8F351977870E3F]:0) > > > [junit4] > at > > > > > > java.security.AccessControlContext.checkPermission(AccessControlContext.j > > > ava:395) > > > [junit4] > at > > > > java.security.AccessController.checkPermission(AccessController.java:559) > > > [junit4] > at > > > java.lang.SecurityManager.checkPermission(SecurityManager.java:549) > > > [junit4] > at > > > java.lang.SecurityManager.checkWrite(SecurityManager.java:979) > > > [junit4] > at java.io.File.canWrite(File.java:785) > > > [junit4] > at > org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:1002) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.createPermissionsDiagnosisString( > > > MiniDFSCluster.java:856) > > > [junit4] > at > > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.j > > > ava:812) > > > [junit4] > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:738) > > > [junit4] > at > > > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:608) > > > [junit4] > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:98) > > > [junit4] > at > > > org.apache.solr.cloud.hdfs.HdfsTestUtil.setupClass(HdfsTestUtil.java:60) > > > [junit4] > at > > > > > > org.apache.solr.store.hdfs.HdfsDirectoryTest.beforeClass(HdfsDirectoryTest. > > > java:62) > > > [junit4] > at java.lang.Thread.run(Thread.java:745)Throwable > #2: > > > com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked > > from > > > SUITE scope at org.apache.solr.store.hdfs.HdfsDirectoryTest: > > > [junit4] > 1) Thread[id=20389, name=IPC Server idle connection > > scanner > > > for port 41610, state=WAITING, group=TGRP-HdfsDirectoryTest] > > > [junit4] > at java.lang.Object.wait(Native Method) > > > [junit4] > at java.lang.Object.wait(Object.java:503) > > > [junit4] > at java.util.TimerThread.mainLoop(Timer.java:526) > > > [junit4] > at java.util.TimerThread.run(Timer.java:505) > > > [junit4] > at > > > __randomizedtesting.SeedInfo.seed([5D8F351977870E3F]:0)Throwable > #3: > > > com.carrotsearch.randomizedtesting.ThreadLeakError: There are still > > zombie > > > threads that couldn't be terminated: > > > [junit4] > 1) Thread[id=20389, name=IPC Server idle connection > > scanner > > > for port 41610, state=WAITING, group=TGRP-HdfsDirectoryTest] > > > [junit4] > at java.lang.Object.wait(Native Method) > > > [junit4] > at java.lang.Object.wait(Object.java:503) > > > [junit4] > at java.util.TimerThread.mainLoop(Timer.java:526) > > > [junit4] > at java.util.TimerThread.run(Timer.java:505) > > > [junit4] > at > > > __randomizedtesting.SeedInfo.seed([5D8F351977870E3F]:0) > > > [junit4] Completed [521/536] on J1 in 33.05s, 0 tests, 3 errors <<< > > FAILURES! > > > > > > [...truncated 64 lines...] > > > BUILD FAILED > > > /export/home/jenkins/workspace/Lucene-Solr-5.x-Solaris/build.xml:785: > > > The following error occurred while executing this line: > > > /export/home/jenkins/workspace/Lucene-Solr-5.x-Solaris/build.xml:729: > > > The following error occurred while executing this line: > > > /export/home/jenkins/workspace/Lucene-Solr-5.x-Solaris/build.xml:59: > > The > > > following error occurred while executing this line: > > > /export/home/jenkins/workspace/Lucene-Solr-5.x- > > > Solaris/solr/build.xml:233: The following error occurred while executing > this > > > line: > > > /export/home/jenkins/workspace/Lucene-Solr-5.x- > Solaris/solr/common- > > > build.xml:524: The following error occurred while executing this line: > > > /export/home/jenkins/workspace/Lucene-Solr-5.x- > > Solaris/lucene/common- > > > build.xml:1452: The following error occurred while executing this line: > > > /export/home/jenkins/workspace/Lucene-Solr-5.x- > > Solaris/lucene/common- > > > build.xml:1006: There were test failures: 536 suites, 2123 tests, 4 suite- > level > > > errors, 108 ignored (34 assumptions) > > > > > > Total time: 77 minutes 51 seconds > > > Build step 'Invoke Ant' marked build as failure > > > Archiving artifacts > > > [WARNINGS] Skipping publisher since build result is FAILURE > > > Recording test results > > > Email was triggered for: Failure - Any > > > Sending email for trigger: Failure - Any > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
