[ https://issues.apache.org/jira/browse/LUCENE-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14563790#comment-14563790 ]
Steve Rowe commented on LUCENE-6507: ------------------------------------ I see an HdfsLockFactoryTest failure on 5.2 after this commit: [http://jenkins.sarowe.net/job/Lucene-Solr-tests-5.2-Java8/3/]. {noformat} [junit4] Suite: org.apache.solr.store.hdfs.HdfsLockFactoryTest [junit4] 2> Creating dataDir: /var/lib/jenkins/jobs/Lucene-Solr-tests-5.2-Java8/workspace/solr/build/solr-core/test/J5/temp/solr.store.hdfs.HdfsLockFactoryTest B48BC404BF6BB3F1-001/init-core-data-001 [junit4] 2> 123149 T2061 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false) [junit4] 2> 124356 T2061 oahu.NativeCodeLoader.<clinit> WARN Unable to load native-hadoop library for your platform... using builtin-java classes where applicable [junit4] 1> Formatting using clusterid: testClusterID [junit4] 2> 125343 T2061 oahmi.MetricsConfig.loadFirst WARN Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties [junit4] 2> 125705 T2061 oml.Slf4jLog.info Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog [junit4] 2> 125714 T2061 oahh.HttpRequestLog.getRequestLog WARN Jetty request log can only be enabled using Log4j [junit4] 2> 126082 T2061 oml.Slf4jLog.info jetty-6.1.26 [junit4] 2> 126200 T2061 oml.Slf4jLog.info Extract jar:file:/var/lib/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_45038_hdfs____jfnzfi/webapp [junit4] 2> 126577 T2061 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet [junit4] 2> 127704 T2061 oml.Slf4jLog.info Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45038 [junit4] 2> 129764 T2061 oahh.HttpRequestLog.getRequestLog WARN Jetty request log can only be enabled using Log4j [junit4] 2> 129777 T2061 oml.Slf4jLog.info jetty-6.1.26 [junit4] 2> 129841 T2061 oml.Slf4jLog.info Extract jar:file:/var/lib/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_60765_datanode____fxr31f/webapp [junit4] 2> 130228 T2061 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet [junit4] 2> 131028 T2061 oml.Slf4jLog.info Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60765 [junit4] 2> 131767 T2061 oahh.HttpRequestLog.getRequestLog WARN Jetty request log can only be enabled using Log4j [junit4] 2> 131769 T2061 oml.Slf4jLog.info jetty-6.1.26 [junit4] 2> 131799 T2061 oml.Slf4jLog.info Extract jar:file:/var/lib/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45594_datanode____xb8eu/webapp [junit4] 2> 132058 T2061 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet [junit4] 2> 132856 T2061 oml.Slf4jLog.info Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45594 [junit4] 2> 135493 T2088 oahhsb.BlockManager.processReport BLOCK* processReport: from storage DS-c7286d21-0c75-425a-b32a-cda888b89811 node DatanodeRegistration(127.0.0.1, datanodeUuid=183b04a7-dc21-4d3e-ad8c-52569c645c0d, infoPort=60765, ipcPort=36559, storageInfo=lv=-56;cid=testClusterID;nsid=349984326;c=0), blocks: 0, hasStaleStorages: true, processing time: 2 msecs [junit4] 2> 135501 T2088 oahhsb.BlockManager.processReport BLOCK* processReport: from storage DS-2cbc54c3-5182-44c8-b80a-a4af3d3bea02 node DatanodeRegistration(127.0.0.1, datanodeUuid=183b04a7-dc21-4d3e-ad8c-52569c645c0d, infoPort=60765, ipcPort=36559, storageInfo=lv=-56;cid=testClusterID;nsid=349984326;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs [junit4] 2> 135493 T2097 oahhsb.BlockManager.processReport BLOCK* processReport: from storage DS-3c1bd938-8f62-4608-a6d4-63f576e970cd node DatanodeRegistration(127.0.0.1, datanodeUuid=3e441a70-defc-4b2f-bf7f-95c351d97a39, infoPort=45594, ipcPort=58299, storageInfo=lv=-56;cid=testClusterID;nsid=349984326;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs [junit4] 2> 135511 T2097 oahhsb.BlockManager.processReport BLOCK* processReport: from storage DS-7b0f0027-583c-462f-9b9c-e6f13ca8a160 node DatanodeRegistration(127.0.0.1, datanodeUuid=3e441a70-defc-4b2f-bf7f-95c351d97a39, infoPort=45594, ipcPort=58299, storageInfo=lv=-56;cid=testClusterID;nsid=349984326;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs [junit4] 2> 135786 T2061 oas.SolrTestCaseJ4.setUp ###Starting testBasic [junit4] 2> 135956 T2061 oassh.HdfsDirectory.<init> WARN The NameNode is in SafeMode - Solr will wait 5 seconds and try again. [junit4] 2> 141161 T2061 oas.SolrTestCaseJ4.tearDown ###Ending testBasic [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HdfsLockFactoryTest -Dtests.method=testBasic -Dtests.seed=B48BC404BF6BB3F1 -Dtests.slow=true -Dtests.locale=de_GR -Dtests.timezone=Antarctica/Vostok -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [junit4] ERROR 5.48s J5 | HdfsLockFactoryTest.testBasic <<< [junit4] > Throwable #1: org.apache.lucene.store.LockObtainFailedException: this lock instance was already obtained [junit4] > at __randomizedtesting.SeedInfo.seed([B48BC404BF6BB3F1:1F71D91160B735DF]:0) [junit4] > at org.apache.solr.store.hdfs.HdfsLockFactory$HdfsLock.obtain(HdfsLockFactory.java:71) [junit4] > at org.apache.solr.store.hdfs.HdfsLockFactoryTest.testBasic(HdfsLockFactoryTest.java:75) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] 2> 141271 T2061 oas.SolrTestCaseJ4.setUp ###Starting testDoubleObtain [junit4] 2> 141393 T2061 oassh.HdfsDirectory.close Closing hdfs directory hdfs://localhost:40796/basedir/lock [junit4] 2> 141394 T2061 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleObtain [junit4] 2> 141394 T2061 oahhsd.DirectoryScanner.shutdown WARN DirectoryScanner: shutdown has been called [junit4] 2> 141435 T2061 oml.Slf4jLog.info Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 [junit4] 2> 141436 T2133 oahh.HttpServer2$SelectChannelConnectorWithSafeStartup.isRunning WARN HttpServer Acceptor: isRunning is false. Rechecking. [junit4] 2> 141437 T2133 oahh.HttpServer2$SelectChannelConnectorWithSafeStartup.isRunning WARN HttpServer Acceptor: isRunning is false [junit4] 2> 141538 T2143 oahhsd.BPServiceActor.offerService WARN BPOfferService for Block pool BP-1715631016-127.0.1.1-1432850493139 (Datanode Uuid 3e441a70-defc-4b2f-bf7f-95c351d97a39) service to localhost/127.0.0.1:40796 interrupted [junit4] 2> 141538 T2143 oahhsd.BPServiceActor.run WARN Ending block pool service for: Block pool BP-1715631016-127.0.1.1-1432850493139 (Datanode Uuid 3e441a70-defc-4b2f-bf7f-95c351d97a39) service to localhost/127.0.0.1:40796 [junit4] 2> 141542 T2061 oahhsd.DirectoryScanner.shutdown WARN DirectoryScanner: shutdown has been called [junit4] 2> 141559 T2061 oml.Slf4jLog.info Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 [junit4] 2> 141559 T2109 oahh.HttpServer2$SelectChannelConnectorWithSafeStartup.isRunning WARN HttpServer Acceptor: isRunning is false. Rechecking. [junit4] 2> 141571 T2109 oahh.HttpServer2$SelectChannelConnectorWithSafeStartup.isRunning WARN HttpServer Acceptor: isRunning is false [junit4] 2> 141580 T2116 oahhsd.BPServiceActor.offerService WARN BPOfferService for Block pool BP-1715631016-127.0.1.1-1432850493139 (Datanode Uuid 183b04a7-dc21-4d3e-ad8c-52569c645c0d) service to localhost/127.0.0.1:40796 interrupted [junit4] 2> 141588 T2116 oahhsd.BPServiceActor.run WARN Ending block pool service for: Block pool BP-1715631016-127.0.1.1-1432850493139 (Datanode Uuid 183b04a7-dc21-4d3e-ad8c-52569c645c0d) service to localhost/127.0.0.1:40796 [junit4] 2> 141598 T2087 oahhsb.DecommissionManager$Monitor.run WARN Monitor interrupted: java.lang.InterruptedException: sleep interrupted [junit4] 2> 141616 T2061 oml.Slf4jLog.info Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 [junit4] 2> 141722 T2061 oahml.MethodMetric$2.snapshot ERROR Error invoking method getBlocksTotal java.lang.reflect.InvocationTargetException [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] 2> at java.lang.reflect.Method.invoke(Method.java:497) [junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111) [junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144) [junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387) [junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415) [junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) [junit4] 2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592) [junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72) [junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68) [junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145) [junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822) [junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720) [junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699) [junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197) [junit4] 2> at org.apache.solr.store.hdfs.HdfsLockFactoryTest.afterClass(HdfsLockFactoryTest.java:52) [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] 2> at java.lang.reflect.Method.invoke(Method.java:497) [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) [junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) [junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) [junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) [junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> Caused by: java.lang.NullPointerException [junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198) [junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291) [junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223) [junit4] 2> ... 54 more [junit4] 2> [junit4] 2> 141730 T2061 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /var/lib/jenkins/jobs/Lucene-Solr-tests-5.2-Java8/workspace/solr/build/solr-core/test/J5/temp/solr.store.hdfs.HdfsLockFactoryTest B48BC404BF6BB3F1-001 [junit4] 2> 18586 T2060 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s). [junit4] 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=de_GR, timezone=Antarctica/Vostok [junit4] 2> NOTE: Linux 4.0.2 amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=16,threads=3,free=274346960,total=521142272 [junit4] 2> NOTE: All tests run in this JVM: [TestDistributedMissingSort, TestRandomMergePolicy, TestSolrConfigHandlerCloud, TestSweetSpotSimilarityFactory, TestSolrJ, TestStressLucene, TestBinaryResponseWriter, TestIndexSearcher, HdfsLockFactoryTest] [junit4] Completed [164/497] on J5 in 28.44s, 2 tests, 1 error <<< FAILURES! {noformat} > NativeFSLock.close() can invalidate other locks > ----------------------------------------------- > > Key: LUCENE-6507 > URL: https://issues.apache.org/jira/browse/LUCENE-6507 > Project: Lucene - Core > Issue Type: Bug > Reporter: Simon Willnauer > Priority: Blocker > Fix For: 4.10.5, 5.2 > > Attachments: LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch, > LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch, > LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch > > > the lock API in Lucene is super trappy since the lock that we return form > this API must first be obtained and if we can't obtain it the lock should not > be closed since we might ie. close the underlying channel in the NativeLock > case which releases all lock for this file on some operating systems. I think > the makeLock method should try to obtain and only return a lock if we > successfully obtained it. Not sure if it's possible everywhere but we should > at least make the documentation clear here. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org