[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.9.0-ea-b60) - Build # 12665 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/12665/ Java: 64bit/jdk1.9.0-ea-b60 -XX:-UseCompressedOops -XX:+UseSerialGC 1 tests failed. FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test Error Message: Captured an uncaught exception in thread: Thread[id=2619, name=parallelCoreAdminExecutor-1404-thread-7, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] Stack Trace: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2619, name=parallelCoreAdminExecutor-1404-thread-7, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] at __randomizedtesting.SeedInfo.seed([8C5E7FB8C8AB60AD:40A406266570D55]:0) Caused by: java.lang.AssertionError: Too many closes on SolrCore at __randomizedtesting.SeedInfo.seed([8C5E7FB8C8AB60AD]:0) at org.apache.solr.core.SolrCore.close(SolrCore.java:1150) at org.apache.solr.common.util.IOUtils.closeQuietly(IOUtils.java:31) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:652) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:611) at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:628) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:213) at org.apache.solr.handler.admin.CoreAdminHandler$ParallelCoreAdminHandlerThread.run(CoreAdminHandler.java:1249) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 10034 lines...] [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest [junit4] 2 Creating dataDir: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest 8C5E7FB8C8AB60AD-001/init-core-data-001 [junit4] 2 246209 T2279 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true) [junit4] 2 246209 T2279 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /gndv/a [junit4] 2 246211 T2279 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 2 246211 T2280 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0 [junit4] 2 246211 T2280 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2 246311 T2279 oasc.ZkTestServer.run start zk server on port:59109 [junit4] 2 246312 T2279 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2 246313 T2279 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2 246316 T2287 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@293a7a58 name:ZooKeeperConnection Watcher:127.0.0.1:59109 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2 246316 T2279 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2 246316 T2279 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2 246317 T2279 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2 246318 T2279 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2 246319 T2279 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2 246335 T2290 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a976749 name:ZooKeeperConnection Watcher:127.0.0.1:59109/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2 246335 T2279 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2 246335 T2279 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2 246336 T2279 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2 246338 T2279 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2 246339 T2279 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2 246340 T2279 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2 246341 T2279 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2 246342 T2279 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2 246343 T2279 oasc.AbstractZkTestCase.putConfig put
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[ https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560693#comment-14560693 ] ASF subversion and git services commented on LUCENE-6481: - Commit 1681942 from [~mikemccand] in branch 'dev/branches/LUCENE-6481' [ https://svn.apache.org/r1681942 ] LUCENE-6481: add nocommit Improve GeoPointField type to only visit high precision boundary terms --- Key: LUCENE-6481 URL: https://issues.apache.org/jira/browse/LUCENE-6481 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Nicholas Knize Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch Current GeoPointField [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges along the space-filling curve that represent a provided bounding box. This determines which terms to visit in the terms dictionary and which to skip. This is suboptimal for large bounding boxes as we may end up visiting all terms (which could be quite large). This incremental improvement is to improve GeoPointField to only visit high precision terms in boundary ranges and use the postings list for ranges that are completely within the target bounding box. A separate improvement is to switch over to auto-prefix and build an Automaton representing the bounding box. That can be tracked in a separate issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6371) Improve Spans payload collection
[ https://issues.apache.org/jira/browse/LUCENE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560771#comment-14560771 ] Alan Woodward commented on LUCENE-6371: --- I'm not sure I'm following what you mean by having two APIs. SpanQuery.createWeight() has exactly the same signature as Query.createWeight() - replacing needsScores with the termcontexts map in the SpanWeight constructor just allows us to pass in the needed information to build the Similarity at the same time as indicating whether or not it should be built at all. Improve Spans payload collection Key: LUCENE-6371 URL: https://issues.apache.org/jira/browse/LUCENE-6371 Project: Lucene - Core Issue Type: Improvement Reporter: Paul Elschot Assignee: Alan Woodward Priority: Minor Fix For: Trunk, 5.3 Attachments: LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch Spin off from LUCENE-6308, see the comments there from around 23 March 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6371) Improve Spans payload collection
[ https://issues.apache.org/jira/browse/LUCENE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560783#comment-14560783 ] Alan Woodward commented on LUCENE-6371: --- Ah, ok. This is more to do with LUCENE-6466. Moving .getSpans() and .extractTerms() to SpanWeight means that we can't collect the termcontexts in the constructor any more, because we can't call them on a partially constructed object. And the TermContexts map was already in the API, but it was in getSpans() rather than in the SpanWeight constructor. You're right that it's messy though. I'm just not sure there's a cleaner way of doing it that doesn't involve completely changing how the SpanScorer works. Improve Spans payload collection Key: LUCENE-6371 URL: https://issues.apache.org/jira/browse/LUCENE-6371 Project: Lucene - Core Issue Type: Improvement Reporter: Paul Elschot Assignee: Alan Woodward Priority: Minor Fix For: Trunk, 5.3 Attachments: LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch Spin off from LUCENE-6308, see the comments there from around 23 March 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[ https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560684#comment-14560684 ] ASF subversion and git services commented on LUCENE-6481: - Commit 1681941 from [~mikemccand] in branch 'dev/branches/LUCENE-6481' [ https://svn.apache.org/r1681941 ] LUCENE-6481: sometimes use smallish bbox for random test Improve GeoPointField type to only visit high precision boundary terms --- Key: LUCENE-6481 URL: https://issues.apache.org/jira/browse/LUCENE-6481 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Nicholas Knize Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch Current GeoPointField [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges along the space-filling curve that represent a provided bounding box. This determines which terms to visit in the terms dictionary and which to skip. This is suboptimal for large bounding boxes as we may end up visiting all terms (which could be quite large). This incremental improvement is to improve GeoPointField to only visit high precision terms in boundary ranges and use the postings list for ranges that are completely within the target bounding box. A separate improvement is to switch over to auto-prefix and build an Automaton representing the bounding box. That can be tracked in a separate issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[
https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560690#comment-14560690
]
Michael McCandless commented on LUCENE-6481:
I committed a small improvement to the test, to sometimes test on a smallish
random sized bbox, and it hits this new failure:
{noformat}
[junit4] Started J0 PID(15031@localhost).
[junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
[junit4] 1 smallBBox=true
[junit4] 1 T0: iter=1 id=3 docID=3 lat=-90.2957562284163
lon=-179.83102465432302 (bbox: lat=-90.84003988710097 TO -89.86198542441215
lon=-180.5339936365967 TO -179.31722107698667) expected true but got: false
deleted?=false query=GeoPointInBBoxQuery: field=geoField: Lower Left:
[-180.5339936365967,-90.84003988710097] Upper Right:
[-179.31722107698667,-89.86198542441215]
[junit4] 2 ??? 27, 2015 12:37:04 PM
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
[junit4] 2 WARNING: Uncaught exception in thread:
Thread[T0,5,TGRP-TestGeoPointQuery]
[junit4] 2 java.lang.AssertionError: wrong result
[junit4] 2at __randomizedtesting.SeedInfo.seed([5]:0)
[junit4] 2at org.junit.Assert.fail(Assert.java:93)
[junit4] 2at
org.apache.lucene.search.TestGeoPointQuery$1._run(TestGeoPointQuery.java:372)
[junit4] 2at
org.apache.lucene.search.TestGeoPointQuery$1.run(TestGeoPointQuery.java:271)
[junit4] 2
[junit4] 2 NOTE: reproduce with: ant test -Dtestcase=TestGeoPointQuery
-Dtests.method=testRandomTiny -Dtests.seed=5 -Dtests.locale=sr_ME
-Dtests.timezone=Europe/Vilnius -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.09s | TestGeoPointQuery.testRandomTiny
{noformat}
I'm not sure why testRandomTiny is hitting all these problems!
Improve GeoPointField type to only visit high precision boundary terms
---
Key: LUCENE-6481
URL: https://issues.apache.org/jira/browse/LUCENE-6481
Project: Lucene - Core
Issue Type: Improvement
Components: core/index
Reporter: Nicholas Knize
Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch,
LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch
Current GeoPointField [LUCENE-6450 |
https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges
along the space-filling curve that represent a provided bounding box. This
determines which terms to visit in the terms dictionary and which to skip.
This is suboptimal for large bounding boxes as we may end up visiting all
terms (which could be quite large).
This incremental improvement is to improve GeoPointField to only visit high
precision terms in boundary ranges and use the postings list for ranges that
are completely within the target bounding box.
A separate improvement is to switch over to auto-prefix and build an
Automaton representing the bounding box. That can be tracked in a separate
issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[ https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560702#comment-14560702 ] ASF subversion and git services commented on LUCENE-6481: - Commit 1681943 from [~mikemccand] in branch 'dev/branches/LUCENE-6481' [ https://svn.apache.org/r1681943 ] LUCENE-6481: woops Improve GeoPointField type to only visit high precision boundary terms --- Key: LUCENE-6481 URL: https://issues.apache.org/jira/browse/LUCENE-6481 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Nicholas Knize Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch Current GeoPointField [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges along the space-filling curve that represent a provided bounding box. This determines which terms to visit in the terms dictionary and which to skip. This is suboptimal for large bounding boxes as we may end up visiting all terms (which could be quite large). This incremental improvement is to improve GeoPointField to only visit high precision terms in boundary ranges and use the postings list for ranges that are completely within the target bounding box. A separate improvement is to switch over to auto-prefix and build an Automaton representing the bounding box. That can be tracked in a separate issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_45) - Build # 4859 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4859/ Java: 32bit/jdk1.8.0_45 -server -XX:+UseG1GC 1 tests failed. FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test Error Message: Captured an uncaught exception in thread: Thread[id=9599, name=parallelCoreAdminExecutor-4953-thread-15, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] Stack Trace: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9599, name=parallelCoreAdminExecutor-4953-thread-15, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] at __randomizedtesting.SeedInfo.seed([FF25C44C4CBB8E2A:7771FB96E247E3D2]:0) Caused by: java.lang.AssertionError: Too many closes on SolrCore at __randomizedtesting.SeedInfo.seed([FF25C44C4CBB8E2A]:0) at org.apache.solr.core.SolrCore.close(SolrCore.java:1150) at org.apache.solr.common.util.IOUtils.closeQuietly(IOUtils.java:31) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:652) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:611) at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:628) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:213) at org.apache.solr.handler.admin.CoreAdminHandler$ParallelCoreAdminHandlerThread.run(CoreAdminHandler.java:1249) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 10605 lines...] [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest [junit4] 2 Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.MultiThreadedOCPTest FF25C44C4CBB8E2A-001\init-core-data-001 [junit4] 2 1768625 T9263 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true) [junit4] 2 1768626 T9263 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: / [junit4] 2 1768627 T9263 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 2 1768628 T9264 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0 [junit4] 2 1768629 T9264 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2 1768701 T9263 oasc.ZkTestServer.run start zk server on port:57319 [junit4] 2 1768729 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2 1768737 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml [junit4] 2 1768742 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2 1768746 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2 1768749 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt [junit4] 2 1768753 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml [junit4] 2 1768759 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2 1768764 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2 1768770 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2 1768775 T9263 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
[jira] [Commented] (LUCENE-6371) Improve Spans payload collection
[
https://issues.apache.org/jira/browse/LUCENE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560775#comment-14560775
]
Robert Muir commented on LUCENE-6371:
-
Sorry, I am not very clear. I don't understand why SpanWeight needs to take
nulls, or TermContext stuff at all.
Why can't it work like today? If someone Span* impl needs to know when scoring
is not needed, add {{boolean needsScores}} to SpanWeight's ctor, rather than
some magic null value. TermContexts map is an impl detail, not an API. Lets not
let it become one, its too messy for that.
Improve Spans payload collection
Key: LUCENE-6371
URL: https://issues.apache.org/jira/browse/LUCENE-6371
Project: Lucene - Core
Issue Type: Improvement
Reporter: Paul Elschot
Assignee: Alan Woodward
Priority: Minor
Fix For: Trunk, 5.3
Attachments: LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch,
LUCENE-6371.patch
Spin off from LUCENE-6308, see the comments there from around 23 March 2015.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7183) SaslZkACLProviderTest reproducible failures due to poor locale blacklisting
[
https://issues.apache.org/jira/browse/SOLR-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560610#comment-14560610
]
ASF subversion and git services commented on SOLR-7183:
---
Commit 1681928 from [~anshumg] in branch 'dev/branches/lucene_solr_5_2'
[ https://svn.apache.org/r1681928 ]
SOLR-7183: Fix Locale blacklisting for Minikdc based tests.(merge from
branch_5x)
SaslZkACLProviderTest reproducible failures due to poor locale blacklisting
---
Key: SOLR-7183
URL: https://issues.apache.org/jira/browse/SOLR-7183
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Assignee: Gregory Chanan
Fix For: 5.2
Attachments: SOLR-7183.patch
SaslZkACLProviderTest has this blacklist of locales...
{code}
// These Locales don't generate dates that are compatibile with Hadoop
MiniKdc.
protected final static ListString brokenLocales =
Arrays.asList(
th_TH_TH_#u-nu-thai,
ja_JP_JP_#u-ca-japanese,
hi_IN);
{code}
..but this list is incomplete -- notably because it only focuses on one
specific Thai variant, and then does a string Locale.toString() comparison.
so at a minimum {{-Dtests.locale=th_TH}} also fails - i suspect there are
other variants that will fail as well
* if there is a bug in Hadoop MiniKdc then that bug should be filed in
jira, and there should be Solr jira that refers to it -- the Solr jira URL
needs to be included her in the test case so developers in the future can
understand the context and have some idea of if/when the third-party lib bug
is fixed
* if we need to work around some Locales because of this bug, then Locale
comparisons need be based on whatever aspects of the Locale are actually
problematic
see for example SOLR-6387 this commit:
https://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/contrib/morphlines-core/src/test/org/apache/solr/morphlines/solr/AbstractSolrMorphlineZkTestBase.java?r1=1618676r2=1618675pathrev=1618676
Or SOLR-6991 + TIKA-1526 this commit:
https://svn.apache.org/viewvc/lucene/dev/branches/lucene_solr_5_0/solr/contrib/extraction/src/test/org/apache/solr/handler/extraction/ExtractingRequestHandlerTest.java?r1=1653708r2=1653707pathrev=1653708
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_45) - Build # 12644 - Failure!
8-real-core Haswell-E with 64G DDR4 memory and a NVMe 750-series SSD. Can run *all* of the Lucene and Solr tests in 10 minutes by running multiple ant jobs in parallel! On Wed, May 27, 2015 at 1:17 AM, Ramkumar R. Aiyengar andyetitmo...@gmail.com wrote: Curious.. sarowe, what's the spec? On 26 May 2015 20:41, Anshum Gupta ans...@anshumgupta.net wrote: The last buch of fixes seems to have fixed this. The tests passed on a Jenkins that had failing tests earlier. Thanks Steve Rowe for lending the super-powerful machine that runs the entire suite in 8 min! I've seen about 20 runs on that box and also runs on Policeman Jenkins with no issues related to this test since the last commit so I've also back-ported this to 5x as well. On Tue, May 26, 2015 at 9:19 AM, Chris Hostetter hossman_luc...@fucit.org wrote: : Right, as I said, we weren't hitting this issue due to our Kerberos conf. : file. i.e. the only thing that was different on our machines as compared to : everyone else and moving that conf file got the tests to fail for me. It : now fails fairly consistently without the patch (from SOLR-7468) and has : been looking good with the patch. that smells like the kind of thing that sould have some assume sanity checks built into it. Given: * the test setups a special env before running the test * the test assumes the specific env will exist * the user's machine may already have env properties set before running ant that affect the expected special env therefore: before the test does the setup of the special env, it should sanity check that the users basic env doesn't have any properties that violate the basic situation. so, hypothetical example based on what little i understand the authentication stuff: if the purpose of a test is to prove that some requests work with (or w/o) kerberos authentication, then before doing any setup of a mock kerberos env (or before setting up a mock situation where no authentication is required), the test should verify that there isn't already an existing kerberos env. (or if possible: unset whatever env/properties define that env) trivial example of a similar situation is the script engine tests -- TestBadConfigs.testBogusScriptEngine: the purpose of the test is to ensure that a solrconfig.xml file that refers to a script engine (by name) which is not installed on the machine will produce an expeted error at Solr init. before doing the Solr init, we have an whitebox assume that asks the JVM directly if a script engine with that name already exists) -Hoss http://www.lucidworks.com/ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Anshum Gupta -- Anshum Gupta
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[ https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560704#comment-14560704 ] Michael McCandless commented on LUCENE-6481: bq. I committed a small improvement to the test, to sometimes test on a smallish random sized bbox, and it hits this new failure: Woops, my bad: test bug. I committed a fix... Improve GeoPointField type to only visit high precision boundary terms --- Key: LUCENE-6481 URL: https://issues.apache.org/jira/browse/LUCENE-6481 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Nicholas Knize Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch Current GeoPointField [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges along the space-filling curve that represent a provided bounding box. This determines which terms to visit in the terms dictionary and which to skip. This is suboptimal for large bounding boxes as we may end up visiting all terms (which could be quite large). This incremental improvement is to improve GeoPointField to only visit high precision terms in boundary ranges and use the postings list for ranges that are completely within the target bounding box. A separate improvement is to switch over to auto-prefix and build an Automaton representing the bounding box. That can be tracked in a separate issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_45) - Build # 12644 - Failure!
Curious.. sarowe, what's the spec? On 26 May 2015 20:41, Anshum Gupta ans...@anshumgupta.net wrote: The last buch of fixes seems to have fixed this. The tests passed on a Jenkins that had failing tests earlier. Thanks Steve Rowe for lending the super-powerful machine that runs the entire suite in 8 min! I've seen about 20 runs on that box and also runs on Policeman Jenkins with no issues related to this test since the last commit so I've also back-ported this to 5x as well. On Tue, May 26, 2015 at 9:19 AM, Chris Hostetter hossman_luc...@fucit.org wrote: : Right, as I said, we weren't hitting this issue due to our Kerberos conf. : file. i.e. the only thing that was different on our machines as compared to : everyone else and moving that conf file got the tests to fail for me. It : now fails fairly consistently without the patch (from SOLR-7468) and has : been looking good with the patch. that smells like the kind of thing that sould have some assume sanity checks built into it. Given: * the test setups a special env before running the test * the test assumes the specific env will exist * the user's machine may already have env properties set before running ant that affect the expected special env therefore: before the test does the setup of the special env, it should sanity check that the users basic env doesn't have any properties that violate the basic situation. so, hypothetical example based on what little i understand the authentication stuff: if the purpose of a test is to prove that some requests work with (or w/o) kerberos authentication, then before doing any setup of a mock kerberos env (or before setting up a mock situation where no authentication is required), the test should verify that there isn't already an existing kerberos env. (or if possible: unset whatever env/properties define that env) trivial example of a similar situation is the script engine tests -- TestBadConfigs.testBogusScriptEngine: the purpose of the test is to ensure that a solrconfig.xml file that refers to a script engine (by name) which is not installed on the machine will produce an expeted error at Solr init. before doing the Solr init, we have an whitebox assume that asks the JVM directly if a script engine with that name already exists) -Hoss http://www.lucidworks.com/ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Anshum Gupta
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_45) - Build # 12841 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/12841/ Java: 64bit/jdk1.8.0_45 -XX:-UseCompressedOops -XX:+UseParallelGC 3 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestSolrConfigHandlerCloud Error Message: ERROR: SolrIndexSearcher opens=281 closes=280 Stack Trace: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=281 closes=280 at __randomizedtesting.SeedInfo.seed([CE5375984EAAF35C]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:497) at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:232) at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestSolrConfigHandlerCloud Error Message: 2 threads leaked from SUITE scope at org.apache.solr.handler.TestSolrConfigHandlerCloud: 1) Thread[id=2806, name=searcherExecutor-1604-thread-1, state=WAITING, group=TGRP-TestSolrConfigHandlerCloud] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)2) Thread[id=2097, name=qtp239723857-2097, state=RUNNABLE, group=TGRP-TestSolrConfigHandlerCloud] at java.util.WeakHashMap.get(WeakHashMap.java:403) at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.calcEtag(HttpCacheHeaderUtil.java:101) at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.doCacheHeaderValidation(HttpCacheHeaderUtil.java:219) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:428) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:105) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) at
[jira] [Commented] (LUCENE-6371) Improve Spans payload collection
[ https://issues.apache.org/jira/browse/LUCENE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560786#comment-14560786 ] Robert Muir commented on LUCENE-6371: - But that does not explain the goal. I want to know the reasoning, what is the tradeoff, what are we gaining by having LUCENE-6466 changes? Thats the big thing I am missing. Are there performance improvements in the benchmark? Otherwise I would rather not have stuff like SpanSimilarity or null TermContexts maps in ctors, unless its really needed for some reason. Improve Spans payload collection Key: LUCENE-6371 URL: https://issues.apache.org/jira/browse/LUCENE-6371 Project: Lucene - Core Issue Type: Improvement Reporter: Paul Elschot Assignee: Alan Woodward Priority: Minor Fix For: Trunk, 5.3 Attachments: LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch Spin off from LUCENE-6308, see the comments there from around 23 March 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6371) Improve Spans payload collection
[ https://issues.apache.org/jira/browse/LUCENE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560756#comment-14560756 ] Robert Muir commented on LUCENE-6371: - But what does it buy? I don't like having two apis to accomplish the same thing. In this case i don't think we really save anything. Improve Spans payload collection Key: LUCENE-6371 URL: https://issues.apache.org/jira/browse/LUCENE-6371 Project: Lucene - Core Issue Type: Improvement Reporter: Paul Elschot Assignee: Alan Woodward Priority: Minor Fix For: Trunk, 5.3 Attachments: LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch Spin off from LUCENE-6308, see the comments there from around 23 March 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6501) Flatten subreader structure in ParallelCompositeReader
[ https://issues.apache.org/jira/browse/LUCENE-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560841#comment-14560841 ] Uwe Schindler commented on LUCENE-6501: --- No problems with testing up to now. I think this is safe for 5.3 (not 5.2) so it can bake longer. Flatten subreader structure in ParallelCompositeReader -- Key: LUCENE-6501 URL: https://issues.apache.org/jira/browse/LUCENE-6501 Project: Lucene - Core Issue Type: Improvement Components: core/index Affects Versions: 5.2 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: Trunk, 5.3 Attachments: LUCENE-6501.patch The current implementation of ParallelCompositeReader reassembles the whole subreader structure of the wrapped reader with ParallelLeafReader and ParallelCompositeReader. This leads to bugs like described in LUCENE-6500. This reaches back to the time when this reader was reimplemented for the first time shortly before release of 4.0. Shortly afterwards, we completely changed our search infrastructure to just call leaves() and working with them. The method getSequentialSubReaders was made protected, just to be implemented by subclasses (like this one). But no external code can ever call it. Also the search API just rely on the baseId in relation to the top-level reader (to correctly present document ids). The structure is completely unimportant. This issue will therefore simplify ParallelCompositeReader to just fetch all LeafReaders and build a flat structure of ParallelLeafReaders from it. This also has the nice side-effect, that only the parallel leaf readers must be equally sized, not their structure. This issue will solve LUCENE-6500 as a side effect. I just opened a new issue for discussion and to have this listed as feature and not bug. In general, we could also hide the ParallelLeafReader class and make it an implementation detail. ParallelCompositeReader would be the only entry point - because people could pass any IndexReader structure in, a single AtomicReader would just produce a CompositeReader with one leaf. We could then also rename it back to ParallelReader (like it was in pre Lucene4). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
NASA/JPL's involvement in Memex: now public
Hey Folks, Just wanted to share publicly some articles recently on NASA and JPL’s involvement in Memex. It’s basically focused around Tika, Nutch and Solr, so keep up the great work on all projects. A sampling of the recent press/articles: E. Landau. Deep Web Search May Help Scientists. NASA Jet Propulsion Laboratory. Web Feature. May 22, 2015. http://www.jpl.nasa.gov/news/news.php?feature=4595 http://www.nasa.gov/jpl/deep-web-search-may-help-scientists R. Chirgwin. JPL joins DARPA’s Memex project: Better search and indexing for space stuff. The Register - UK. May 27, 2015. http://www.theregister.co.uk/2015/05/27/jpl_joins_darpas_memex_project/ J. Lendino. NASA, DARPA collaborating on Deep Web search to analyze spacecraft data. ExtremeTech. May 26, 2015. http://goo.gl/iRAFXC M. Prigg. Nasa joins US government project to create ’Google for the dark net’ that could uncover cyber criminals, paedophiles and drug dealers in the online underworld. Daily Mail - UK. May 25, 2015. http://goo.gl/xMj2hH J. Wolman. Knee-deep in data. The Positive. May 27, 2015. http://thepositive.com/knee-deep-in-data/ Scientists will benefit more from Deep Web Searches. GreenAtom. May 26, 2015. http://goo.gl/UqDJAN NASA to put science face on scary DARPA project. TRTWorld.com. May 26, 2015. http://goo.gl/HJPv5O Kitware Participates in DARPA Memex Program, Developing Software to Address Complex Search Problems. PR News, May 23, 2015. http://www.prweb.com/releases/2015/05/prweb12741017.htm Any questions let me know. Cheers, Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
[jira] [Commented] (LUCENE-6501) Flatten subreader structure in ParallelCompositeReader
[ https://issues.apache.org/jira/browse/LUCENE-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560886#comment-14560886 ] ASF subversion and git services commented on LUCENE-6501: - Commit 1682000 from [~thetaphi] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1682000 ] Merged revision(s) 1681998 from lucene/dev/trunk: LUCENE-6501: Flatten subreader structure in ParallelCompositeReader (fixes close listener bug LUCENE-6500) Flatten subreader structure in ParallelCompositeReader -- Key: LUCENE-6501 URL: https://issues.apache.org/jira/browse/LUCENE-6501 Project: Lucene - Core Issue Type: Improvement Components: core/index Affects Versions: 5.2 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: Trunk, 5.3 Attachments: LUCENE-6501.patch The current implementation of ParallelCompositeReader reassembles the whole subreader structure of the wrapped reader with ParallelLeafReader and ParallelCompositeReader. This leads to bugs like described in LUCENE-6500. This reaches back to the time when this reader was reimplemented for the first time shortly before release of 4.0. Shortly afterwards, we completely changed our search infrastructure to just call leaves() and working with them. The method getSequentialSubReaders was made protected, just to be implemented by subclasses (like this one). But no external code can ever call it. Also the search API just rely on the baseId in relation to the top-level reader (to correctly present document ids). The structure is completely unimportant. This issue will therefore simplify ParallelCompositeReader to just fetch all LeafReaders and build a flat structure of ParallelLeafReaders from it. This also has the nice side-effect, that only the parallel leaf readers must be equally sized, not their structure. This issue will solve LUCENE-6500 as a side effect. I just opened a new issue for discussion and to have this listed as feature and not bug. In general, we could also hide the ParallelLeafReader class and make it an implementation detail. ParallelCompositeReader would be the only entry point - because people could pass any IndexReader structure in, a single AtomicReader would just produce a CompositeReader with one leaf. We could then also rename it back to ParallelReader (like it was in pre Lucene4). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6487) Add WGS84 capability to geo3d support
[ https://issues.apache.org/jira/browse/LUCENE-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560979#comment-14560979 ] David Smiley commented on LUCENE-6487: -- This looks ready to me. I do have a couple lingering thoughts (_not blockers_): * Might it make sense for Vector to to have it's x, y, and z, plus GeoPoint's magnitude as floats instead of doubles? I was just thinking the objects are comparatively heavy compared to a lat-lon based Point. * It'd be nice if there was some random round-trip tests from lat lon to GeoPoint to back again, checking the result is within a tolerance. [~nknize] do you have an interest in reviewing the Geo3d lucene6487 branch with respect to WGS84 support? Add WGS84 capability to geo3d support - Key: LUCENE-6487 URL: https://issues.apache.org/jira/browse/LUCENE-6487 Project: Lucene - Core Issue Type: Improvement Components: modules/spatial Reporter: Karl Wright Attachments: LUCENE-6487.patch, LUCENE-6487.patch, LUCENE-6487.patch, LUCENE-6487.patch WGS84 compatibility has been requested for geo3d. This involves working with an ellipsoid rather than a unit sphere. The general formula for an ellipsoid is: x^2/a^2 + y^2/b^2 + z^2/c^2 = 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6501) Flatten subreader structure in ParallelCompositeReader
[ https://issues.apache.org/jira/browse/LUCENE-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560875#comment-14560875 ] ASF subversion and git services commented on LUCENE-6501: - Commit 1681998 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1681998 ] LUCENE-6501: Flatten subreader structure in ParallelCompositeReader (fixes close listener bug LUCENE-6500) Flatten subreader structure in ParallelCompositeReader -- Key: LUCENE-6501 URL: https://issues.apache.org/jira/browse/LUCENE-6501 Project: Lucene - Core Issue Type: Improvement Components: core/index Affects Versions: 5.2 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: Trunk, 5.3 Attachments: LUCENE-6501.patch The current implementation of ParallelCompositeReader reassembles the whole subreader structure of the wrapped reader with ParallelLeafReader and ParallelCompositeReader. This leads to bugs like described in LUCENE-6500. This reaches back to the time when this reader was reimplemented for the first time shortly before release of 4.0. Shortly afterwards, we completely changed our search infrastructure to just call leaves() and working with them. The method getSequentialSubReaders was made protected, just to be implemented by subclasses (like this one). But no external code can ever call it. Also the search API just rely on the baseId in relation to the top-level reader (to correctly present document ids). The structure is completely unimportant. This issue will therefore simplify ParallelCompositeReader to just fetch all LeafReaders and build a flat structure of ParallelLeafReaders from it. This also has the nice side-effect, that only the parallel leaf readers must be equally sized, not their structure. This issue will solve LUCENE-6500 as a side effect. I just opened a new issue for discussion and to have this listed as feature and not bug. In general, we could also hide the ParallelLeafReader class and make it an implementation detail. ParallelCompositeReader would be the only entry point - because people could pass any IndexReader structure in, a single AtomicReader would just produce a CompositeReader with one leaf. We could then also rename it back to ParallelReader (like it was in pre Lucene4). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6500) ParallelCompositeReader does not always call closed listeners
[ https://issues.apache.org/jira/browse/LUCENE-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560878#comment-14560878 ] ASF subversion and git services commented on LUCENE-6500: - Commit 1681998 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1681998 ] LUCENE-6501: Flatten subreader structure in ParallelCompositeReader (fixes close listener bug LUCENE-6500) ParallelCompositeReader does not always call closed listeners - Key: LUCENE-6500 URL: https://issues.apache.org/jira/browse/LUCENE-6500 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-6500.patch CompositeParallelReader misses to call closed listeners when the reader which is provided at construction time does not wrap leaf readers directly, such as a multi reader over directory readers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: NASA/JPL's involvement in Memex: now public
The name Memex is, I think, already in use by a company that was bought out by Autonomy some years ago, and thence by HP. I presume this is *not* what you are talking about? Thanks, Karl On Wed, May 27, 2015 at 8:26 AM, Mattmann, Chris A (3980) chris.a.mattm...@jpl.nasa.gov wrote: Hey Folks, Just wanted to share publicly some articles recently on NASA and JPL’s involvement in Memex. It’s basically focused around Tika, Nutch and Solr, so keep up the great work on all projects. A sampling of the recent press/articles: E. Landau. Deep Web Search May Help Scientists. NASA Jet Propulsion Laboratory. Web Feature. May 22, 2015. http://www.jpl.nasa.gov/news/news.php?feature=4595 http://www.nasa.gov/jpl/deep-web-search-may-help-scientists R. Chirgwin. JPL joins DARPA’s Memex project: Better search and indexing for space stuff. The Register - UK. May 27, 2015. http://www.theregister.co.uk/2015/05/27/jpl_joins_darpas_memex_project/ J. Lendino. NASA, DARPA collaborating on Deep Web search to analyze spacecraft data. ExtremeTech. May 26, 2015. http://goo.gl/iRAFXC M. Prigg. Nasa joins US government project to create ’Google for the dark net’ that could uncover cyber criminals, paedophiles and drug dealers in the online underworld. Daily Mail - UK. May 25, 2015. http://goo.gl/xMj2hH J. Wolman. Knee-deep in data. The Positive. May 27, 2015. http://thepositive.com/knee-deep-in-data/ Scientists will benefit more from Deep Web Searches. GreenAtom. May 26, 2015. http://goo.gl/UqDJAN NASA to put science face on scary DARPA project. TRTWorld.com. May 26, 2015. http://goo.gl/HJPv5O Kitware Participates in DARPA Memex Program, Developing Software to Address Complex Search Problems. PR News, May 23, 2015. http://www.prweb.com/releases/2015/05/prweb12741017.htm Any questions let me know. Cheers, Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
[jira] [Commented] (SOLR-7146) MiniSolrCloudCluster based tests can fail with ZooKeeperException NoNode for /live_nodes
[
https://issues.apache.org/jira/browse/SOLR-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560903#comment-14560903
]
ASF subversion and git services commented on SOLR-7146:
---
Commit 1682002 from sha...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1682002 ]
SOLR-7146: MiniSolrCloudCluster based tests can fail with ZooKeeperException
NoNode for /live_nodes
MiniSolrCloudCluster based tests can fail with ZooKeeperException NoNode for
/live_nodes
Key: SOLR-7146
URL: https://issues.apache.org/jira/browse/SOLR-7146
Project: Solr
Issue Type: Bug
Components: SolrCloud, Tests
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Priority: Minor
Fix For: Trunk, 5.2
Attachments: SOLR-7146.patch, SOLR-7146v2.patch
MiniSolrCloudCluster based tests can fail with the following exception:
{code}
org.apache.solr.common.cloud.ZooKeeperException:
at
__randomizedtesting.SeedInfo.seed([3F3D838A8ADC9385:F153ADFBF163EC6D]:0)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:463)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:763)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:752)
at
org.apache.solr.cloud.MiniSolrCloudCluster.createCollection(MiniSolrCloudCluster.java:193)
at
org.apache.solr.handler.component.TestTwoPhaseDistributedQuery.testNoExtraFieldsRequestedFromShardsInPhaseOne(TestTwoPhaseDistributedQuery.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
Re: [JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_45) - Build # 12644 - Failure!
Oh baby, I need a new computer soon... On Wed, May 27, 2015 at 5:08 AM Anshum Gupta ans...@anshumgupta.net wrote: 8-real-core Haswell-E with 64G DDR4 memory and a NVMe 750-series SSD. Can run *all* of the Lucene and Solr tests in 10 minutes by running multiple ant jobs in parallel! On Wed, May 27, 2015 at 1:17 AM, Ramkumar R. Aiyengar andyetitmo...@gmail.com wrote: Curious.. sarowe, what's the spec? On 26 May 2015 20:41, Anshum Gupta ans...@anshumgupta.net wrote: The last buch of fixes seems to have fixed this. The tests passed on a Jenkins that had failing tests earlier. Thanks Steve Rowe for lending the super-powerful machine that runs the entire suite in 8 min! I've seen about 20 runs on that box and also runs on Policeman Jenkins with no issues related to this test since the last commit so I've also back-ported this to 5x as well. On Tue, May 26, 2015 at 9:19 AM, Chris Hostetter hossman_luc...@fucit.org wrote: : Right, as I said, we weren't hitting this issue due to our Kerberos conf. : file. i.e. the only thing that was different on our machines as compared to : everyone else and moving that conf file got the tests to fail for me. It : now fails fairly consistently without the patch (from SOLR-7468) and has : been looking good with the patch. that smells like the kind of thing that sould have some assume sanity checks built into it. Given: * the test setups a special env before running the test * the test assumes the specific env will exist * the user's machine may already have env properties set before running ant that affect the expected special env therefore: before the test does the setup of the special env, it should sanity check that the users basic env doesn't have any properties that violate the basic situation. so, hypothetical example based on what little i understand the authentication stuff: if the purpose of a test is to prove that some requests work with (or w/o) kerberos authentication, then before doing any setup of a mock kerberos env (or before setting up a mock situation where no authentication is required), the test should verify that there isn't already an existing kerberos env. (or if possible: unset whatever env/properties define that env) trivial example of a similar situation is the script engine tests -- TestBadConfigs.testBogusScriptEngine: the purpose of the test is to ensure that a solrconfig.xml file that refers to a script engine (by name) which is not installed on the machine will produce an expeted error at Solr init. before doing the Solr init, we have an whitebox assume that asks the JVM directly if a script engine with that name already exists) -Hoss http://www.lucidworks.com/ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Anshum Gupta -- Anshum Gupta
[jira] [Created] (LUCENE-6504) implement norms with random access API
Robert Muir created LUCENE-6504: --- Summary: implement norms with random access API Key: LUCENE-6504 URL: https://issues.apache.org/jira/browse/LUCENE-6504 Project: Lucene - Core Issue Type: Improvement Reporter: Robert Muir We added this api in LUCENE-5729 but we never explored implementing norms with it. These are generally the largest consumer of heap memory and often a real hassle for users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7599) Remove cruft from SolrCloud tests
Shalin Shekhar Mangar created SOLR-7599: --- Summary: Remove cruft from SolrCloud tests Key: SOLR-7599 URL: https://issues.apache.org/jira/browse/SOLR-7599 Project: Solr Issue Type: Task Components: SolrCloud, Tests Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: Trunk, 5.3 I see many tests which blindly have distribSetUp and distribTearDown methods setting a variety of options and system properties that aren't required anymore. This is because some base test classes have been refactored such that these options are redundant. In other cases, people have copied the structure of tests blindly instead of understanding what each parameter does. Let's try to remove the unnecessary config params from such tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_45) - Build # 12644 - Failure!
SSD: http://www.newegg.com/Product/Product.aspx?Item=N82E16820167299 CPU: http://www.newegg.com/Product/Product.aspx?Item=N82E16819117404 M/B: http://www.newegg.com/Product/Product.aspx?Item=N82E16813132518 RAM: http://www.newegg.com/Product/Product.aspx?Item=N82E16820231820 The mem wasn’t listed as supported by the mobo manufacturer, and it isn’t detected at its full speed (2800MHz), so currently running at 2400 (“overclocked” from detected 2100 I think). CPU isn’t overclocked from stock 3GHz, but I got a liquid cooler, thinking I’d experiment (haven’t much yet). Even without overclocking the fans spin faster when all the cores are busy, and it’s quite the little space heater. I installed Debian 8, but had to fix the installer in a couple places because it didn’t know about the new NVMe device naming scheme: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=785147 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=785149 I also upgraded to the 4.0 Linux kernel, since Debian 8 ships with the 3.16 kernel, and 3.19 contains a bunch of NVMe improvements. And I turned “swappiness down to zero (thanks to Mike: http://blog.mikemccandless.com/2011/04/just-say-no-to-swapping.html) after seeing a bunch of test stalls while running the Lucene monster tests with 4 JVMs (takes about 2 hours, but I can get it down to 90 minutes or so by splitting the two tests in Test2BSortedDocValues out into their own suites - I’ll make an issue). Steve On May 27, 2015, at 5:08 AM, Anshum Gupta ans...@anshumgupta.net wrote: 8-real-core Haswell-E with 64G DDR4 memory and a NVMe 750-series SSD. Can run *all* of the Lucene and Solr tests in 10 minutes by running multiple ant jobs in parallel! On Wed, May 27, 2015 at 1:17 AM, Ramkumar R. Aiyengar andyetitmo...@gmail.com wrote: Curious.. sarowe, what's the spec? On 26 May 2015 20:41, Anshum Gupta ans...@anshumgupta.net wrote: The last buch of fixes seems to have fixed this. The tests passed on a Jenkins that had failing tests earlier. Thanks Steve Rowe for lending the super-powerful machine that runs the entire suite in 8 min! I've seen about 20 runs on that box and also runs on Policeman Jenkins with no issues related to this test since the last commit so I've also back-ported this to 5x as well. On Tue, May 26, 2015 at 9:19 AM, Chris Hostetter hossman_luc...@fucit.org wrote: : Right, as I said, we weren't hitting this issue due to our Kerberos conf. : file. i.e. the only thing that was different on our machines as compared to : everyone else and moving that conf file got the tests to fail for me. It : now fails fairly consistently without the patch (from SOLR-7468) and has : been looking good with the patch. that smells like the kind of thing that sould have some assume sanity checks built into it. Given: * the test setups a special env before running the test * the test assumes the specific env will exist * the user's machine may already have env properties set before running ant that affect the expected special env therefore: before the test does the setup of the special env, it should sanity check that the users basic env doesn't have any properties that violate the basic situation. so, hypothetical example based on what little i understand the authentication stuff: if the purpose of a test is to prove that some requests work with (or w/o) kerberos authentication, then before doing any setup of a mock kerberos env (or before setting up a mock situation where no authentication is required), the test should verify that there isn't already an existing kerberos env. (or if possible: unset whatever env/properties define that env) trivial example of a similar situation is the script engine tests -- TestBadConfigs.testBogusScriptEngine: the purpose of the test is to ensure that a solrconfig.xml file that refers to a script engine (by name) which is not installed on the machine will produce an expeted error at Solr init. before doing the Solr init, we have an whitebox assume that asks the JVM directly if a script engine with that name already exists) -Hoss http://www.lucidworks.com/ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Anshum Gupta -- Anshum Gupta - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7590) Finish and improve MDC logging support.
[ https://issues.apache.org/jira/browse/SOLR-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561005#comment-14561005 ] ASF subversion and git services commented on SOLR-7590: --- Commit 1682031 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1682031 ] SOLR-7590: Finish and improve MDC context logging support. Finish and improve MDC logging support. --- Key: SOLR-7590 URL: https://issues.apache.org/jira/browse/SOLR-7590 Project: Solr Issue Type: Improvement Reporter: Mark Miller Assignee: Mark Miller Fix For: Trunk, 5.3 Attachments: SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4506) [solr4.0.0] many index.{date} dir in replication node
[
https://issues.apache.org/jira/browse/SOLR-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561026#comment-14561026
]
Timothy Potter commented on SOLR-4506:
--
Hi [~markrmil...@gmail.com], I'd like to take this one up as I need it for some
other issue I'm working on. Let me know if you have any updated code or ideas
on this otherwise, feel free to assign over to me and I'll work on it for 5.3.
Thanks
[solr4.0.0] many index.{date} dir in replication node
--
Key: SOLR-4506
URL: https://issues.apache.org/jira/browse/SOLR-4506
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.0
Environment: the solr4.0 runs on suse11.
mem:32G
cpu:16 cores
Reporter: zhuojunjian
Assignee: Mark Miller
Priority: Minor
Fix For: 4.9, Trunk
Original Estimate: 12h
Remaining Estimate: 12h
in our test,we used solrcloud feature in solr4.0(version detail
:4.0.0.2012.10.06.03.04.33).
the solrcloud configuration is 3 shards and 2 replications each shard.
we found that there are over than 25 dirs which named index.{date} in one
replication node belonging to shard 3.
for example:
index.2013021725864 index.20130218012211880 index.20130218015714713
index.20130218023101958 index.20130218030424083 tlog
index.20130218005648324 index.20130218012751078 index.20130218020141293
the issue seems like SOLR-1781. but it is fixed in 4.0-BETA,5.0.
so is solr4.0 ? if it is fixed too in solr4.0, why we find the issue again ?
what can I do?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6504) implement norms with random access API
[ https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561025#comment-14561025 ] Uwe Schindler commented on LUCENE-6504: --- There are some unrelated changes in ByteBufferIndexInput: You removed the offset=0 implementation. Did you find out that this optimization for offset=0 brings no additional performance? implement norms with random access API -- Key: LUCENE-6504 URL: https://issues.apache.org/jira/browse/LUCENE-6504 Project: Lucene - Core Issue Type: Improvement Reporter: Robert Muir Attachments: LUCENE-6504.patch We added this api in LUCENE-5729 but we never explored implementing norms with it. These are generally the largest consumer of heap memory and often a real hassle for users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7587) TestSpellCheckResponse stalled and never timed out -- possible VersionBucket bug? (5.2 branch)
[
https://issues.apache.org/jira/browse/SOLR-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560969#comment-14560969
]
ASF subversion and git services commented on SOLR-7587:
---
Commit 1682019 from [~thelabdude] in branch 'dev/branches/lucene_solr_5_2'
[ https://svn.apache.org/r1682019 ]
SOLR-7587: Move the call to seed version buckets to before buffering updates
during core construction
TestSpellCheckResponse stalled and never timed out -- possible VersionBucket
bug? (5.2 branch)
--
Key: SOLR-7587
URL: https://issues.apache.org/jira/browse/SOLR-7587
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Assignee: Timothy Potter
Priority: Blocker
Fix For: Trunk, 5.2
Attachments: SOLR-7587.patch, SOLR-7587_minor_refactor.patch,
jstack.1.txt, jstack.2.txt, junit4-J0-20150522_181244_599.events,
junit4-J0-20150522_181244_599.spill, junit4-J0-20150522_181244_599.suites
On the 5.2 branch (r1681250), I encountered a solrj test stalled for over 110
minutes before i finally killed it...
{noformat}
[junit4] Suite: org.apache.solr.common.util.TestRetryUtil
[junit4] Completed [55/60] on J1 in 1.04s, 1 test
[junit4]
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:14:56, stalled for
121s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:15:56, stalled for
181s at: TestSpellCheckResponse.testSpellCheckResponse
...
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:00:56, stalled for
6481s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:01:56, stalled for
6541s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:02:56, stalled for
6601s at: TestSpellCheckResponse.testSpellCheckResponse
{noformat}
I'll attach some jstack output as well as all the temp files from the J0
runner.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7599) Remove cruft from SolrCloud tests
[ https://issues.apache.org/jira/browse/SOLR-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560968#comment-14560968 ] Shalin Shekhar Mangar commented on SOLR-7599: - Here's a list of stuff that should be looked at: # Does a test really need distribSetUp and distribTearDown methods? # Usefulness of useJettyDataDir # solr.xml.persist # The need to change the value for DirectUpdateHandler2.commitOnClose. Only a few tests really need it. # Setting system properties such as System.setProperty(numShards, Integer.toString(sliceCount)); # checkCreatedVsState -- looks like it is always false across all tests. Remove cruft from SolrCloud tests - Key: SOLR-7599 URL: https://issues.apache.org/jira/browse/SOLR-7599 Project: Solr Issue Type: Task Components: SolrCloud, Tests Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: Trunk, 5.3 I see many tests which blindly have distribSetUp and distribTearDown methods setting a variety of options and system properties that aren't required anymore. This is because some base test classes have been refactored such that these options are redundant. In other cases, people have copied the structure of tests blindly instead of understanding what each parameter does. Let's try to remove the unnecessary config params from such tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-7587) TestSpellCheckResponse stalled and never timed out -- possible VersionBucket bug? (5.2 branch)
[
https://issues.apache.org/jira/browse/SOLR-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Potter resolved SOLR-7587.
--
Resolution: Fixed
Fix Version/s: Trunk
TestSpellCheckResponse stalled and never timed out -- possible VersionBucket
bug? (5.2 branch)
--
Key: SOLR-7587
URL: https://issues.apache.org/jira/browse/SOLR-7587
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Assignee: Timothy Potter
Priority: Blocker
Fix For: Trunk, 5.2
Attachments: SOLR-7587.patch, SOLR-7587_minor_refactor.patch,
jstack.1.txt, jstack.2.txt, junit4-J0-20150522_181244_599.events,
junit4-J0-20150522_181244_599.spill, junit4-J0-20150522_181244_599.suites
On the 5.2 branch (r1681250), I encountered a solrj test stalled for over 110
minutes before i finally killed it...
{noformat}
[junit4] Suite: org.apache.solr.common.util.TestRetryUtil
[junit4] Completed [55/60] on J1 in 1.04s, 1 test
[junit4]
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:14:56, stalled for
121s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:15:56, stalled for
181s at: TestSpellCheckResponse.testSpellCheckResponse
...
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:00:56, stalled for
6481s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:01:56, stalled for
6541s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:02:56, stalled for
6601s at: TestSpellCheckResponse.testSpellCheckResponse
{noformat}
I'll attach some jstack output as well as all the temp files from the J0
runner.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7590) Finish and improve MDC logging support.
[ https://issues.apache.org/jira/browse/SOLR-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560970#comment-14560970 ] ASF subversion and git services commented on SOLR-7590: --- Commit 1682020 from [~markrmil...@gmail.com] in branch 'dev/trunk' [ https://svn.apache.org/r1682020 ] SOLR-7590: Finish and improve MDC context logging support. Finish and improve MDC logging support. --- Key: SOLR-7590 URL: https://issues.apache.org/jira/browse/SOLR-7590 Project: Solr Issue Type: Improvement Reporter: Mark Miller Assignee: Mark Miller Fix For: Trunk, 5.3 Attachments: SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6487) Add WGS84 capability to geo3d support
[ https://issues.apache.org/jira/browse/LUCENE-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560971#comment-14560971 ] ASF subversion and git services commented on LUCENE-6487: - Commit 1682021 from [~dsmiley] in branch 'dev/branches/lucene6487' [ https://svn.apache.org/r1682021 ] LUCENE-6487: Geo3D with WGS84 patch from Karl: GeoPoint.getLat getLon. Add WGS84 capability to geo3d support - Key: LUCENE-6487 URL: https://issues.apache.org/jira/browse/LUCENE-6487 Project: Lucene - Core Issue Type: Improvement Components: modules/spatial Reporter: Karl Wright Attachments: LUCENE-6487.patch, LUCENE-6487.patch, LUCENE-6487.patch, LUCENE-6487.patch WGS84 compatibility has been requested for geo3d. This involves working with an ellipsoid rather than a unit sphere. The general formula for an ellipsoid is: x^2/a^2 + y^2/b^2 + z^2/c^2 = 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[ https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560985#comment-14560985 ] ASF subversion and git services commented on LUCENE-6481: - Commit 1682029 from [~mikemccand] in branch 'dev/branches/LUCENE-6481' [ https://svn.apache.org/r1682029 ] LUCENE-6481: validate lat/lon passed to the queries Improve GeoPointField type to only visit high precision boundary terms --- Key: LUCENE-6481 URL: https://issues.apache.org/jira/browse/LUCENE-6481 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Nicholas Knize Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch Current GeoPointField [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges along the space-filling curve that represent a provided bounding box. This determines which terms to visit in the terms dictionary and which to skip. This is suboptimal for large bounding boxes as we may end up visiting all terms (which could be quite large). This incremental improvement is to improve GeoPointField to only visit high precision terms in boundary ranges and use the postings list for ranges that are completely within the target bounding box. A separate improvement is to switch over to auto-prefix and build an Automaton representing the bounding box. That can be tracked in a separate issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_45) - Build # 12843 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/12843/ Java: 32bit/jdk1.8.0_45 -client -XX:+UseConcMarkSweepGC 1 tests failed. FAILED: org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls Error Message: Shard split did not complete. Last recorded state: running expected:[completed] but was:[running] Stack Trace: org.junit.ComparisonFailure: Shard split did not complete. Last recorded state: running expected:[completed] but was:[running] at __randomizedtesting.SeedInfo.seed([E204DA36B31551FE:BA605657B57FF92A]:0) at org.junit.Assert.assertEquals(Assert.java:125) at org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls(CollectionsAPIAsyncDistributedZkTest.java:101) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (LUCENE-6504) implement norms with random access API
[ https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560951#comment-14560951 ] Michael McCandless commented on LUCENE-6504: +1 implement norms with random access API -- Key: LUCENE-6504 URL: https://issues.apache.org/jira/browse/LUCENE-6504 Project: Lucene - Core Issue Type: Improvement Reporter: Robert Muir Attachments: LUCENE-6504.patch We added this api in LUCENE-5729 but we never explored implementing norms with it. These are generally the largest consumer of heap memory and often a real hassle for users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7587) TestSpellCheckResponse stalled and never timed out -- possible VersionBucket bug? (5.2 branch)
[
https://issues.apache.org/jira/browse/SOLR-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560964#comment-14560964
]
ASF subversion and git services commented on SOLR-7587:
---
Commit 1682017 from [~thelabdude] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1682017 ]
SOLR-7587: Move the call to seed version buckets to before buffering updates
during core construction
TestSpellCheckResponse stalled and never timed out -- possible VersionBucket
bug? (5.2 branch)
--
Key: SOLR-7587
URL: https://issues.apache.org/jira/browse/SOLR-7587
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Assignee: Timothy Potter
Priority: Blocker
Fix For: 5.2
Attachments: SOLR-7587.patch, SOLR-7587_minor_refactor.patch,
jstack.1.txt, jstack.2.txt, junit4-J0-20150522_181244_599.events,
junit4-J0-20150522_181244_599.spill, junit4-J0-20150522_181244_599.suites
On the 5.2 branch (r1681250), I encountered a solrj test stalled for over 110
minutes before i finally killed it...
{noformat}
[junit4] Suite: org.apache.solr.common.util.TestRetryUtil
[junit4] Completed [55/60] on J1 in 1.04s, 1 test
[junit4]
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:14:56, stalled for
121s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:15:56, stalled for
181s at: TestSpellCheckResponse.testSpellCheckResponse
...
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:00:56, stalled for
6481s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:01:56, stalled for
6541s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:02:56, stalled for
6601s at: TestSpellCheckResponse.testSpellCheckResponse
{noformat}
I'll attach some jstack output as well as all the temp files from the J0
runner.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6371) Improve Spans payload collection
[ https://issues.apache.org/jira/browse/LUCENE-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560857#comment-14560857 ] Alan Woodward commented on LUCENE-6371: --- It's the same logic as https://issues.apache.org/jira/browse/LUCENE-6425 really. Spans should only be pulled from a SpanQuery after it has been rewritten against a searcher, so it makes sense that they should be on SpanWeight rather than SpanQuery. And it simplifies pulling Spans for use in things like highlighting (or payload collection, as here) - before you had to rewrite, extract terms, build the termcontexts map and then call getSpans(); now you just call IndexSearcher.getNormalizedWeight(), cast to a SpanWeight and call getSpans. Improve Spans payload collection Key: LUCENE-6371 URL: https://issues.apache.org/jira/browse/LUCENE-6371 Project: Lucene - Core Issue Type: Improvement Reporter: Paul Elschot Assignee: Alan Woodward Priority: Minor Fix For: Trunk, 5.3 Attachments: LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch, LUCENE-6371.patch Spin off from LUCENE-6308, see the comments there from around 23 March 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6500) ParallelCompositeReader does not always call closed listeners
[ https://issues.apache.org/jira/browse/LUCENE-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler resolved LUCENE-6500. --- Resolution: Fixed Fix Version/s: 5.3 Trunk Assignee: Uwe Schindler (was: Adrien Grand) Fixed through subreader flattening in LUCENE-6501. Thanks Adrien for reporting! ParallelCompositeReader does not always call closed listeners - Key: LUCENE-6500 URL: https://issues.apache.org/jira/browse/LUCENE-6500 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Uwe Schindler Priority: Minor Fix For: Trunk, 5.3 Attachments: LUCENE-6500.patch CompositeParallelReader misses to call closed listeners when the reader which is provided at construction time does not wrap leaf readers directly, such as a multi reader over directory readers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7587) TestSpellCheckResponse stalled and never timed out -- possible VersionBucket bug? (5.2 branch)
[
https://issues.apache.org/jira/browse/SOLR-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560961#comment-14560961
]
ASF subversion and git services commented on SOLR-7587:
---
Commit 1682016 from [~thelabdude] in branch 'dev/trunk'
[ https://svn.apache.org/r1682016 ]
SOLR-7587: Move the call to seed version buckets to before buffering updates
during core construction
TestSpellCheckResponse stalled and never timed out -- possible VersionBucket
bug? (5.2 branch)
--
Key: SOLR-7587
URL: https://issues.apache.org/jira/browse/SOLR-7587
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Assignee: Timothy Potter
Priority: Blocker
Fix For: 5.2
Attachments: SOLR-7587.patch, SOLR-7587_minor_refactor.patch,
jstack.1.txt, jstack.2.txt, junit4-J0-20150522_181244_599.events,
junit4-J0-20150522_181244_599.spill, junit4-J0-20150522_181244_599.suites
On the 5.2 branch (r1681250), I encountered a solrj test stalled for over 110
minutes before i finally killed it...
{noformat}
[junit4] Suite: org.apache.solr.common.util.TestRetryUtil
[junit4] Completed [55/60] on J1 in 1.04s, 1 test
[junit4]
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:14:56, stalled for
121s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:15:56, stalled for
181s at: TestSpellCheckResponse.testSpellCheckResponse
...
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:00:56, stalled for
6481s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:01:56, stalled for
6541s at: TestSpellCheckResponse.testSpellCheckResponse
[junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:02:56, stalled for
6601s at: TestSpellCheckResponse.testSpellCheckResponse
{noformat}
I'll attach some jstack output as well as all the temp files from the J0
runner.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6504) implement norms with random access API
[ https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560987#comment-14560987 ] David Smiley commented on LUCENE-6504: -- Nice. Before/now, AFAIK norms take 1 byte per field per doc of heap. I looked over the patch briefly; does this essentially put norms off-heap? And I noticed it has more fidelity than a single byte; up to 8 in fact. Does this patch also bring in accurate norms or is something else required to enable actually utilize that? implement norms with random access API -- Key: LUCENE-6504 URL: https://issues.apache.org/jira/browse/LUCENE-6504 Project: Lucene - Core Issue Type: Improvement Reporter: Robert Muir Attachments: LUCENE-6504.patch We added this api in LUCENE-5729 but we never explored implementing norms with it. These are generally the largest consumer of heap memory and often a real hassle for users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6501) Flatten subreader structure in ParallelCompositeReader
[ https://issues.apache.org/jira/browse/LUCENE-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler resolved LUCENE-6501. --- Resolution: Fixed Flatten subreader structure in ParallelCompositeReader -- Key: LUCENE-6501 URL: https://issues.apache.org/jira/browse/LUCENE-6501 Project: Lucene - Core Issue Type: Improvement Components: core/index Affects Versions: 5.2 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: Trunk, 5.3 Attachments: LUCENE-6501.patch The current implementation of ParallelCompositeReader reassembles the whole subreader structure of the wrapped reader with ParallelLeafReader and ParallelCompositeReader. This leads to bugs like described in LUCENE-6500. This reaches back to the time when this reader was reimplemented for the first time shortly before release of 4.0. Shortly afterwards, we completely changed our search infrastructure to just call leaves() and working with them. The method getSequentialSubReaders was made protected, just to be implemented by subclasses (like this one). But no external code can ever call it. Also the search API just rely on the baseId in relation to the top-level reader (to correctly present document ids). The structure is completely unimportant. This issue will therefore simplify ParallelCompositeReader to just fetch all LeafReaders and build a flat structure of ParallelLeafReaders from it. This also has the nice side-effect, that only the parallel leaf readers must be equally sized, not their structure. This issue will solve LUCENE-6500 as a side effect. I just opened a new issue for discussion and to have this listed as feature and not bug. In general, we could also hide the ParallelLeafReader class and make it an implementation detail. ParallelCompositeReader would be the only entry point - because people could pass any IndexReader structure in, a single AtomicReader would just produce a CompositeReader with one leaf. We could then also rename it back to ParallelReader (like it was in pre Lucene4). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6500) ParallelCompositeReader does not always call closed listeners
[ https://issues.apache.org/jira/browse/LUCENE-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560887#comment-14560887 ] ASF subversion and git services commented on LUCENE-6500: - Commit 1682000 from [~thetaphi] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1682000 ] Merged revision(s) 1681998 from lucene/dev/trunk: LUCENE-6501: Flatten subreader structure in ParallelCompositeReader (fixes close listener bug LUCENE-6500) ParallelCompositeReader does not always call closed listeners - Key: LUCENE-6500 URL: https://issues.apache.org/jira/browse/LUCENE-6500 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-6500.patch CompositeParallelReader misses to call closed listeners when the reader which is provided at construction time does not wrap leaf readers directly, such as a multi reader over directory readers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: NASA/JPL's involvement in Memex: now public
Hi Karl, Memex is a DARPA initiative to build a next generation search engine capability for the Deep/Dark web. It’s of course a play on the famous Vannevar Bush paper in 1945: http://www.darpa.mil/newsevents/releases/2014/02/09.aspx The articles I shared below explain NASA and JPL’s involvement below in addition our use of Tika, Nutch, and Lucene/Solr. Cheers, Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ -Original Message- From: Karl Wright daddy...@gmail.com Reply-To: d...@nutch.apache.org d...@nutch.apache.org Date: Wednesday, May 27, 2015 at 8:33 AM To: dev@lucene.apache.org dev@lucene.apache.org Cc: d...@tika.apache.org d...@tika.apache.org, d...@nutch.apache.org d...@nutch.apache.org Subject: Re: NASA/JPL's involvement in Memex: now public The name Memex is, I think, already in use by a company that was bought out by Autonomy some years ago, and thence by HP. I presume this is *not* what you are talking about? Thanks, Karl On Wed, May 27, 2015 at 8:26 AM, Mattmann, Chris A (3980) chris.a.mattm...@jpl.nasa.gov wrote: Hey Folks, Just wanted to share publicly some articles recently on NASA and JPL’s involvement in Memex. It’s basically focused around Tika, Nutch and Solr, so keep up the great work on all projects. A sampling of the recent press/articles: E. Landau. Deep Web Search May Help Scientists. NASA Jet Propulsion Laboratory. Web Feature. May 22, 2015. http://www.jpl.nasa.gov/news/news.php?feature=4595 http://www.nasa.gov/jpl/deep-web-search-may-help-scientists R. Chirgwin. JPL joins DARPA’s Memex project: Better search and indexing for space stuff. The Register - UK. May 27, 2015. http://www.theregister.co.uk/2015/05/27/jpl_joins_darpas_memex_project/ J. Lendino. NASA, DARPA collaborating on Deep Web search to analyze spacecraft data. ExtremeTech. May 26, 2015. http://goo.gl/iRAFXC M. Prigg. Nasa joins US government project to create ’Google for the dark net’ that could uncover cyber criminals, paedophiles and drug dealers in the online underworld. Daily Mail - UK. May 25, 2015. http://goo.gl/xMj2hH J. Wolman. Knee-deep in data. The Positive. May 27, 2015. http://thepositive.com/knee-deep-in-data/ Scientists will benefit more from Deep Web Searches. GreenAtom. May 26, 2015. http://goo.gl/UqDJAN NASA to put science face on scary DARPA project. TRTWorld.com. May 26, 2015. http://goo.gl/HJPv5O Kitware Participates in DARPA Memex Program, Developing Software to Address Complex Search Problems. PR News, May 23, 2015. http://www.prweb.com/releases/2015/05/prweb12741017.htm Any questions let me know. Cheers, Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7146) MiniSolrCloudCluster based tests can fail with ZooKeeperException NoNode for /live_nodes
[
https://issues.apache.org/jira/browse/SOLR-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560906#comment-14560906
]
ASF subversion and git services commented on SOLR-7146:
---
Commit 1682003 from sha...@apache.org in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1682003 ]
SOLR-7146: MiniSolrCloudCluster based tests can fail with ZooKeeperException
NoNode for /live_nodes
MiniSolrCloudCluster based tests can fail with ZooKeeperException NoNode for
/live_nodes
Key: SOLR-7146
URL: https://issues.apache.org/jira/browse/SOLR-7146
Project: Solr
Issue Type: Bug
Components: SolrCloud, Tests
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Priority: Minor
Fix For: Trunk, 5.2
Attachments: SOLR-7146.patch, SOLR-7146v2.patch
MiniSolrCloudCluster based tests can fail with the following exception:
{code}
org.apache.solr.common.cloud.ZooKeeperException:
at
__randomizedtesting.SeedInfo.seed([3F3D838A8ADC9385:F153ADFBF163EC6D]:0)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:463)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:763)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:752)
at
org.apache.solr.cloud.MiniSolrCloudCluster.createCollection(MiniSolrCloudCluster.java:193)
at
org.apache.solr.handler.component.TestTwoPhaseDistributedQuery.testNoExtraFieldsRequestedFromShardsInPhaseOne(TestTwoPhaseDistributedQuery.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
[jira] [Resolved] (SOLR-7146) MiniSolrCloudCluster based tests can fail with ZooKeeperException NoNode for /live_nodes
[
https://issues.apache.org/jira/browse/SOLR-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar resolved SOLR-7146.
-
I increased the timeout to a minute and changed the logic to break immediately
if numservers is reached.
Thanks Vamsee!
MiniSolrCloudCluster based tests can fail with ZooKeeperException NoNode for
/live_nodes
Key: SOLR-7146
URL: https://issues.apache.org/jira/browse/SOLR-7146
Project: Solr
Issue Type: Bug
Components: SolrCloud, Tests
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Priority: Minor
Fix For: Trunk, 5.3
Attachments: SOLR-7146.patch, SOLR-7146v2.patch
MiniSolrCloudCluster based tests can fail with the following exception:
{code}
org.apache.solr.common.cloud.ZooKeeperException:
at
__randomizedtesting.SeedInfo.seed([3F3D838A8ADC9385:F153ADFBF163EC6D]:0)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:463)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:763)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:752)
at
org.apache.solr.cloud.MiniSolrCloudCluster.createCollection(MiniSolrCloudCluster.java:193)
at
org.apache.solr.handler.component.TestTwoPhaseDistributedQuery.testNoExtraFieldsRequestedFromShardsInPhaseOne(TestTwoPhaseDistributedQuery.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
[jira] [Updated] (LUCENE-6504) implement norms with random access API
[
https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-6504:
Attachment: LUCENE-6504.patch
Here is a patch. We just use these methods to get signed values as a very
simplistic compression.
Generally speaking the performance seems ok, I think its the right tradeoff.
{noformat}
Report after iter 19:
Chart saved to out.png... (wd: /home/rmuir/workspace/util/src/python)
Task QPS trunk StdDev QPS patch StdDev
Pct diff
LowTerm 983.09 (3.8%) 903.02 (4.6%)
-8.1% ( -15% -0%)
HighSpanNear 158.40 (2.7%) 148.01 (2.0%)
-6.6% ( -11% - -1%)
OrNotHighMed 213.20 (3.9%) 199.50 (2.8%)
-6.4% ( -12% -0%)
OrNotHighLow 1170.07 (2.7%) 1134.55 (2.4%)
-3.0% ( -7% -2%)
AndHighHigh 87.91 (1.9%) 86.21 (1.8%)
-1.9% ( -5% -1%)
IntNRQ8.64 (5.6%)8.48 (8.0%)
-1.8% ( -14% - 12%)
AndHighMed 123.04 (1.8%) 120.85 (1.7%)
-1.8% ( -5% -1%)
Fuzzy2 60.37 (1.3%) 59.35 (1.8%)
-1.7% ( -4% -1%)
Wildcard 44.77 (3.2%) 44.06 (4.6%)
-1.6% ( -9% -6%)
MedSpanNear 150.07 (3.1%) 148.15 (2.6%)
-1.3% ( -6% -4%)
LowSpanNear 30.53 (1.2%) 30.15 (1.5%)
-1.2% ( -3% -1%)
LowPhrase 33.89 (2.1%) 33.49 (3.7%)
-1.2% ( -6% -4%)
Prefix3 210.10 (3.8%) 207.61 (5.3%)
-1.2% ( -9% -8%)
AndHighLow 1180.40 (2.1%) 1166.82 (2.5%)
-1.2% ( -5% -3%)
Respell 81.67 (1.5%) 81.41 (2.4%)
-0.3% ( -4% -3%)
Fuzzy1 97.84 (1.3%) 97.64 (1.7%)
-0.2% ( -3% -2%)
LowSloppyPhrase 120.00 (3.0%) 120.42 (3.0%)
0.4% ( -5% -6%)
MedPhrase 263.96 (4.8%) 265.05 (7.1%)
0.4% ( -10% - 12%)
MedSloppyPhrase 15.97 (4.5%) 16.08 (4.6%)
0.7% ( -8% - 10%)
MedTerm 175.47 (4.4%) 177.64 (5.7%)
1.2% ( -8% - 11%)
HighPhrase 17.80 (6.3%) 18.20 (9.3%)
2.2% ( -12% - 19%)
OrHighMed 54.02 (6.8%) 56.18 (7.7%)
4.0% ( -9% - 19%)
OrHighLow 52.35 (6.9%) 54.61 (7.7%)
4.3% ( -9% - 20%)
HighSloppyPhrase 12.85 (10.3%) 13.41 (11.9%)
4.4% ( -16% - 29%)
OrHighHigh 25.34 (7.2%) 26.77 (8.3%)
5.6% ( -9% - 22%)
HighTerm 119.17 (4.7%) 128.45 (7.1%)
7.8% ( -3% - 20%)
OrHighNotLow 110.06 (6.4%) 119.06 (7.0%)
8.2% ( -4% - 23%)
OrHighNotMed 91.03 (6.1%) 98.91 (6.4%)
8.7% ( -3% - 22%)
OrNotHighHigh 51.53 (5.6%) 56.04 (6.4%)
8.8% ( -3% - 21%)
OrHighNotHigh 30.45 (5.7%) 33.27 (6.2%)
9.2% ( -2% - 22%)
{noformat}
implement norms with random access API
--
Key: LUCENE-6504
URL: https://issues.apache.org/jira/browse/LUCENE-6504
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
Attachments: LUCENE-6504.patch
We added this api in LUCENE-5729 but we never explored implementing norms
with it. These are generally the largest consumer of heap memory and often a
real hassle for users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7555) Display total space and available space in Admin
[ https://issues.apache.org/jira/browse/SOLR-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Pugh updated SOLR-7555: Attachment: SOLR-7555-display_disk_space_v2.patch Here is an updated patch. I moved the method spaceAvailable and spaceTotal to the abstract Directory class, and then implemented it for the various classes that extended it. I don't love that this change touches the Lucene package, as I've never really gone deep there before, and it feels rather invasive! I also changed the format of the response to mimic the how the memory hands back both human formatted and raw data. Display total space and available space in Admin Key: SOLR-7555 URL: https://issues.apache.org/jira/browse/SOLR-7555 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 5.1 Reporter: Eric Pugh Assignee: Erik Hatcher Priority: Minor Fix For: 5.2 Attachments: SOLR-7555-display_disk_space.patch, SOLR-7555-display_disk_space_v2.patch, SOLR-7555.patch Frequently I have access to the Solr Admin console, but not the underlying server, and I'm curious how much space remains available. This little patch exposes total Volume size as well as the usable space remaining: !https://monosnap.com/file/VqlReekCFwpK6utI3lP18fbPqrGI4b.png! I'm not sure if this is the best place to put this, as every shard will share the same data, so maybe it should be on the top level Dashboard? Also not sure what to call the fields! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7555) Display total space and available space in Admin
[ https://issues.apache.org/jira/browse/SOLR-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561441#comment-14561441 ] Eric Pugh commented on SOLR-7555: - [~mariusneo]this was perfect! Display total space and available space in Admin Key: SOLR-7555 URL: https://issues.apache.org/jira/browse/SOLR-7555 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 5.1 Reporter: Eric Pugh Assignee: Erik Hatcher Priority: Minor Fix For: 5.2 Attachments: SOLR-7555-display_disk_space.patch, SOLR-7555-display_disk_space_v2.patch, SOLR-7555.patch Frequently I have access to the Solr Admin console, but not the underlying server, and I'm curious how much space remains available. This little patch exposes total Volume size as well as the usable space remaining: !https://monosnap.com/file/VqlReekCFwpK6utI3lP18fbPqrGI4b.png! I'm not sure if this is the best place to put this, as every shard will share the same data, so maybe it should be on the top level Dashboard? Also not sure what to call the fields! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-6820) The sync on the VersionInfo bucket in DistributedUpdateProcesser#addDocument appears to be a large bottleneck when using replication.
[
https://issues.apache.org/jira/browse/SOLR-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Potter reopened SOLR-6820:
--
Copy-and-paste error in the configset solrconfig.xml files, the int parameter
is missing the name!
{code}
updateLog
str name=dir${solr.ulog.dir:}/str
int name=${solr.ulog.numVersionBuckets:65536}/int
/updateLog
{code}
but should be:
{code}
updateLog
str name=dir${solr.ulog.dir:}/str
int name=numVersionBuckets${solr.ulog.numVersionBuckets:65536}/int
/updateLog
{code}
It doesn't look like this causes any failures but looks bad.
The sync on the VersionInfo bucket in DistributedUpdateProcesser#addDocument
appears to be a large bottleneck when using replication.
-
Key: SOLR-6820
URL: https://issues.apache.org/jira/browse/SOLR-6820
Project: Solr
Issue Type: Sub-task
Components: SolrCloud
Reporter: Mark Miller
Assignee: Timothy Potter
Fix For: Trunk, 5.2
Attachments: SOLR-6820.patch, threads.png
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Building the first RC for Lucene/Solr 5.2.0
Hi Anshum, I re-opened SOLR-6820 due to a minor issue with a parameter in solrconfig.xml missing the name attribute. It doesn't cause any failures, but looks bad ... so I don't think it warrants a re-spin but if we go to another RC, then I'll get that in ... if not, I'll close SOLR-6820 and open a separate ticket for that problem. Cheers, Tim On Wed, May 27, 2015 at 9:52 AM, Anshum Gupta ans...@anshumgupta.net wrote: FYI, I've started the process to build the first RC for Lucene/Solr 5.2.0 -- Anshum Gupta - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Hmm, bug (schemaless/schema api)?
On May 27, 2015, at 1:40 PM, Gus Heck gus.h...@gmail.com wrote: What you say is what I had generally understood to be the case in the past with regular static schemas, but I had hoped something newer and more flexible was implied with the advent of an API, and the schemaless configuration. Although there is a caveat at the top of the wiki page, statements like When modifying the schema with the REST API, a core reload will automatically occur in order for the changes to be available immediately. had given me hope. Guess you haven't got that far after all. Thanks, I added a(nother) warning to that sentence. Steve - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Building the first RC for Lucene/Solr 5.2.0
Sure, sounds good to me. I'm running the smoketester now so the RC should be out for vote right after. On Wed, May 27, 2015 at 11:41 AM, Timothy Potter thelabd...@gmail.com wrote: Hi Anshum, I re-opened SOLR-6820 due to a minor issue with a parameter in solrconfig.xml missing the name attribute. It doesn't cause any failures, but looks bad ... so I don't think it warrants a re-spin but if we go to another RC, then I'll get that in ... if not, I'll close SOLR-6820 and open a separate ticket for that problem. Cheers, Tim On Wed, May 27, 2015 at 9:52 AM, Anshum Gupta ans...@anshumgupta.net wrote: FYI, I've started the process to build the first RC for Lucene/Solr 5.2.0 -- Anshum Gupta - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Anshum Gupta
[jira] [Commented] (SOLR-7344) Use two thread pools, one for internal requests and one for external, to avoid distributed deadlock and decrease the number of threads that need to be created.
[ https://issues.apache.org/jira/browse/SOLR-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561532#comment-14561532 ] Hrishikesh Gadre commented on SOLR-7344: [~yo...@apache.org] I have an alternate proposal similar to your earlier idea of having a logical partition of a jetty worker pool without requiring a separate endpoint. I believe this would also solve the problem of request forwarding. The idea here is to implement a custom Servlet filter similar to [Jetty QoSFilter|http://www.eclipse.org/jetty/documentation/current/qos-filter.html] which would reserve a certain capacity of the worker thread-pool for internal and external requests. The advantages of this approach are as follows, - We will still get to use the thread-per-request model when the number of requests (of a specific type) are less than the capacity reserved for that request type. All the additional requests will be suspended until atleast one thread is available for processing. - We will not require extensive changes to the SolrDispatcherFilter code thereby reducing the possibility of introducing new bugs. - Since this new filter needs to be added to the solr webapp, folks can opt out if they want (Please see the earlier comment from [~andyetitmoves]). Note even for this approach, we need to implement a way to identify an internal request (e.g. adding an extra request header/param). Now regarding the request forwarding, I have following change in mind. Let's assume that a client C has sent a request to node A for a collection/core hosted by node B. - The initial request from client C will be categorized as an external request. - Node A will figure out that it does not host the requested collection/core. It will figure out the correct Solr server (node B in this case) and send another request to node B. Here we will ensure that this request is also categorized as an external request (i.e. not to add the request header/param even though its a server-server communication). - Node B will process the forwarded request as if it were an external request and return results to node A. - Node A will return results to client C. Use two thread pools, one for internal requests and one for external, to avoid distributed deadlock and decrease the number of threads that need to be created. --- Key: SOLR-7344 URL: https://issues.apache.org/jira/browse/SOLR-7344 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Mark Miller Attachments: SOLR-7344.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7555) Display total space and available space in Admin
[ https://issues.apache.org/jira/browse/SOLR-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561545#comment-14561545 ] Upayavira commented on SOLR-7555: - Extending the Directory class seems like a pretty major step - one that should get far wider coverage in its own ticket, I'd say. However, a simpler way to do it would be to have a DiskSpaceAwareDirectory interface that provides the necessary methods. Then, the API can only offer disk usage values from Directory objects that implement that interface. Dunno if that pattern is used at all in Lucene/Solr though. Display total space and available space in Admin Key: SOLR-7555 URL: https://issues.apache.org/jira/browse/SOLR-7555 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 5.1 Reporter: Eric Pugh Assignee: Erik Hatcher Priority: Minor Fix For: 5.2 Attachments: SOLR-7555-display_disk_space.patch, SOLR-7555-display_disk_space_v2.patch, SOLR-7555.patch Frequently I have access to the Solr Admin console, but not the underlying server, and I'm curious how much space remains available. This little patch exposes total Volume size as well as the usable space remaining: !https://monosnap.com/file/VqlReekCFwpK6utI3lP18fbPqrGI4b.png! I'm not sure if this is the best place to put this, as every shard will share the same data, so maybe it should be on the top level Dashboard? Also not sure what to call the fields! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Building the first RC for Lucene/Solr 5.2.0
FYI, I've started the process to build the first RC for Lucene/Solr 5.2.0 -- Anshum Gupta
[jira] [Commented] (LUCENE-6487) Add WGS84 capability to geo3d support
[ https://issues.apache.org/jira/browse/LUCENE-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561176#comment-14561176 ] Karl Wright commented on LUCENE-6487: - Simple round-trip tests are trivial, but I won't get to one today, and I agree that it is not a blocker for commit. We can create a separate ticket if you go ahead and merge. re doubles vs. floats: Well, the actual (x,y,z) value needs to be accurate to within 1e-12 in order to properly specify various planes etc throughout the module. So I'd be cautious about deliberately reducing precision, since geopoint extends vector and plane does too. We can experiment but... once again, I think a separate ticket would be in order. Add WGS84 capability to geo3d support - Key: LUCENE-6487 URL: https://issues.apache.org/jira/browse/LUCENE-6487 Project: Lucene - Core Issue Type: Improvement Components: modules/spatial Reporter: Karl Wright Attachments: LUCENE-6487.patch, LUCENE-6487.patch, LUCENE-6487.patch, LUCENE-6487.patch WGS84 compatibility has been requested for geo3d. This involves working with an ellipsoid rather than a unit sphere. The general formula for an ellipsoid is: x^2/a^2 + y^2/b^2 + z^2/c^2 = 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7590) Finish and improve MDC logging support.
[ https://issues.apache.org/jira/browse/SOLR-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561136#comment-14561136 ] ASF subversion and git services commented on SOLR-7590: --- Commit 1682057 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1682057 ] SOLR-7590: Java8 code - Java7 for branch_5x. Finish and improve MDC logging support. --- Key: SOLR-7590 URL: https://issues.apache.org/jira/browse/SOLR-7590 Project: Solr Issue Type: Improvement Reporter: Mark Miller Assignee: Mark Miller Fix For: Trunk, 5.3 Attachments: SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7361) Main Jetty thread blocked by core loading delays HTTP listener from binding if core loading is slow
[
https://issues.apache.org/jira/browse/SOLR-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561158#comment-14561158
]
ASF subversion and git services commented on SOLR-7361:
---
Commit 1682060 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1682060 ]
SOLR-7361: Slow loading SolrCores should not hold up all other SolrCores that
have finished loading from serving requests.
Main Jetty thread blocked by core loading delays HTTP listener from binding
if core loading is slow
---
Key: SOLR-7361
URL: https://issues.apache.org/jira/browse/SOLR-7361
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Timothy Potter
Assignee: Mark Miller
Fix For: Trunk, 5.2
Attachments: SOLR-7361.patch, SOLR-7361.patch, SOLR-7361.patch,
SOLR-7361.patch, SOLR-7361.patch, SOLR-7361.patch, SOLR-7361.patch,
SOLR-7361.patch
During server startup, the CoreContainer uses an ExecutorService to load
cores in multiple back-ground threads but then blocks until cores are loaded,
see: CoreContainer#load around line 290 on trunk (invokeAll). From the
JavaDoc on that method, we have:
{quote}
Executes the given tasks, returning a list of Futures holding their status
and results when all complete. Future.isDone() is true for each element of
the returned list.
{quote}
In other words, this is a blocking call.
This delays the Jetty HTTP listener from binding and accepting requests until
all cores are loaded. Do we need to block the main thread?
Also, prior to this happening, the node is registered as a live node in ZK,
which makes it a candidate for receiving requests from the Overseer, such as
to service a create collection request. The problem of course is that the
node listed in /live_nodes isn't accepting requests yet. So we either need to
unblock the main thread during server loading or maybe wait longer before we
register as a live node ... not sure which is the better way forward?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7361) Main Jetty thread blocked by core loading delays HTTP listener from binding if core loading is slow
[
https://issues.apache.org/jira/browse/SOLR-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561175#comment-14561175
]
ASF subversion and git services commented on SOLR-7361:
---
Commit 1682065 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1682065 ]
SOLR-7361: Slow loading SolrCores should not hold up all other SolrCores that
have finished loading from serving requests.
Main Jetty thread blocked by core loading delays HTTP listener from binding
if core loading is slow
---
Key: SOLR-7361
URL: https://issues.apache.org/jira/browse/SOLR-7361
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Timothy Potter
Assignee: Mark Miller
Fix For: Trunk, 5.2
Attachments: SOLR-7361.patch, SOLR-7361.patch, SOLR-7361.patch,
SOLR-7361.patch, SOLR-7361.patch, SOLR-7361.patch, SOLR-7361.patch,
SOLR-7361.patch
During server startup, the CoreContainer uses an ExecutorService to load
cores in multiple back-ground threads but then blocks until cores are loaded,
see: CoreContainer#load around line 290 on trunk (invokeAll). From the
JavaDoc on that method, we have:
{quote}
Executes the given tasks, returning a list of Futures holding their status
and results when all complete. Future.isDone() is true for each element of
the returned list.
{quote}
In other words, this is a blocking call.
This delays the Jetty HTTP listener from binding and accepting requests until
all cores are loaded. Do we need to block the main thread?
Also, prior to this happening, the node is registered as a live node in ZK,
which makes it a candidate for receiving requests from the Overseer, such as
to service a create collection request. The problem of course is that the
node listed in /live_nodes isn't accepting requests yet. So we either need to
unblock the main thread during server loading or maybe wait longer before we
register as a live node ... not sure which is the better way forward?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Hmm, bug?
So I did the following (seemingly) reasonable set of steps... 1. Set up a Solr Cloud instance with mutable managed schema (data driven config) 2. Tossed a couple docs in just to see it work. 3. Tried to sort on 'title', realized title had been guessed as multivalued. 4. Changed the field to type=string (as opposed to strings) using schema api 5. Tried to sort on title The result is - trace: java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for field 'title' (expected=SORTED). Use UninvertingReader or index with docvalues.\n\tat org.apache.lucene.index.DocValues.checkField(DocValues.java:208) - Resending the curl command to re-index the docs hangs However, I have not played with schemaless or the schema api before, so maybe I've missed something, and my steps are not reasonable, and thus this sanity check before I file a bug. -Gus -- http://www.the111shift.com
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[ https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561056#comment-14561056 ] ASF subversion and git services commented on LUCENE-6481: - Commit 1682043 from [~mikemccand] in branch 'dev/branches/LUCENE-6481' [ https://svn.apache.org/r1682043 ] LUCENE-6481: add nocommit about query slowness for large bboxes; improve javadocs Improve GeoPointField type to only visit high precision boundary terms --- Key: LUCENE-6481 URL: https://issues.apache.org/jira/browse/LUCENE-6481 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Nicholas Knize Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch Current GeoPointField [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges along the space-filling curve that represent a provided bounding box. This determines which terms to visit in the terms dictionary and which to skip. This is suboptimal for large bounding boxes as we may end up visiting all terms (which could be quite large). This incremental improvement is to improve GeoPointField to only visit high precision terms in boundary ranges and use the postings list for ranges that are completely within the target bounding box. A separate improvement is to switch over to auto-prefix and build an Automaton representing the bounding box. That can be tracked in a separate issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4506) [solr4.0.0] many index.{date} dir in replication node
[
https://issues.apache.org/jira/browse/SOLR-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561089#comment-14561089
]
Mark Miller commented on SOLR-4506:
---
Fire away - fell off my radar - can't remember 2013 :)
[solr4.0.0] many index.{date} dir in replication node
--
Key: SOLR-4506
URL: https://issues.apache.org/jira/browse/SOLR-4506
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.0
Environment: the solr4.0 runs on suse11.
mem:32G
cpu:16 cores
Reporter: zhuojunjian
Assignee: Timothy Potter
Priority: Minor
Fix For: 4.9, Trunk
Original Estimate: 12h
Remaining Estimate: 12h
in our test,we used solrcloud feature in solr4.0(version detail
:4.0.0.2012.10.06.03.04.33).
the solrcloud configuration is 3 shards and 2 replications each shard.
we found that there are over than 25 dirs which named index.{date} in one
replication node belonging to shard 3.
for example:
index.2013021725864 index.20130218012211880 index.20130218015714713
index.20130218023101958 index.20130218030424083 tlog
index.20130218005648324 index.20130218012751078 index.20130218020141293
the issue seems like SOLR-1781. but it is fixed in 4.0-BETA,5.0.
so is solr4.0 ? if it is fixed too in solr4.0, why we find the issue again ?
what can I do?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4506) [solr4.0.0] many index.{date} dir in replication node
[
https://issues.apache.org/jira/browse/SOLR-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Miller updated SOLR-4506:
--
Assignee: Timothy Potter (was: Mark Miller)
[solr4.0.0] many index.{date} dir in replication node
--
Key: SOLR-4506
URL: https://issues.apache.org/jira/browse/SOLR-4506
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.0
Environment: the solr4.0 runs on suse11.
mem:32G
cpu:16 cores
Reporter: zhuojunjian
Assignee: Timothy Potter
Priority: Minor
Fix For: 4.9, Trunk
Original Estimate: 12h
Remaining Estimate: 12h
in our test,we used solrcloud feature in solr4.0(version detail
:4.0.0.2012.10.06.03.04.33).
the solrcloud configuration is 3 shards and 2 replications each shard.
we found that there are over than 25 dirs which named index.{date} in one
replication node belonging to shard 3.
for example:
index.2013021725864 index.20130218012211880 index.20130218015714713
index.20130218023101958 index.20130218030424083 tlog
index.20130218005648324 index.20130218012751078 index.20130218020141293
the issue seems like SOLR-1781. but it is fixed in 4.0-BETA,5.0.
so is solr4.0 ? if it is fixed too in solr4.0, why we find the issue again ?
what can I do?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6504) implement norms with random access API
[ https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561095#comment-14561095 ] Uwe Schindler commented on LUCENE-6504: --- bq. Its not unrelated, i did extensive benchmarking. That optimization is a trap, allowing too many low level implementations (3) once the index gets large. OK. That's fine. I never liked that offset=0 specialization, i think back at that time we felt it might be a good idea. I don't know the issue number anymore. You also removed the negative check in the new Multi implementation, but I think this was done because of performance, too. I am not happy with that, because it no longer throws Exception if you try to access stuff off-slice. Maybe we can add an assert instead? implement norms with random access API -- Key: LUCENE-6504 URL: https://issues.apache.org/jira/browse/LUCENE-6504 Project: Lucene - Core Issue Type: Improvement Reporter: Robert Muir Attachments: LUCENE-6504.patch We added this api in LUCENE-5729 but we never explored implementing norms with it. These are generally the largest consumer of heap memory and often a real hassle for users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 694 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/694/ 4 tests failed. REGRESSION: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test Error Message: The Monkey ran for over 20 seconds and no jetties were stopped - this is worth investigating! Stack Trace: java.lang.AssertionError: The Monkey ran for over 20 seconds and no jetties were stopped - this is worth investigating! at __randomizedtesting.SeedInfo.seed([BF0290866116A66E:3756AF5CCFEACB96]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:537) at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:152) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
Re: svn commit: r1682031 [1/2] - in /lucene/dev/branches/branch_5x: ./ solr/ solr/core/ solr/core/src/java/org/apache/solr/ solr/core/src/java/org/apache/solr/cloud/ solr/core/src/java/org/apache/solr
Java7 compilation borked?:
[javac] Compiling 807 source files to
/var/lib/jenkins/jobs/Solr-core-tests-5x-Java7/workspace/solr/build/solr-core/classes/java
[javac]
/var/lib/jenkins/jobs/Solr-core-tests-5x-Java7/workspace/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java:26:
error: package java.util.function does not exist
[javac] import java.util.function.Supplier;
[javac] ^
[javac]
/var/lib/jenkins/jobs/Solr-core-tests-5x-Java7/workspace/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java:42:
error: cannot find symbol
[javac] private static ThreadLocalInteger CALL_DEPTH =
ThreadLocal.withInitial(new SupplierInteger() {
[javac]
^
[javac] symbol: class Supplier
[javac] location: class MDCLoggingContext
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 2 errors
Steve
On May 27, 2015, at 9:58 AM, markrmil...@apache.org wrote:
Author: markrmiller
Date: Wed May 27 13:58:30 2015
New Revision: 1682031
URL: http://svn.apache.org/r1682031
Log:
SOLR-7590: Finish and improve MDC context logging support.
Added:
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java
- copied unchanged from r1682020,
lucene/dev/trunk/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java
Removed:
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/SolrLogFormatter.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/logging/MDCUtils.java
Modified:
lucene/dev/branches/branch_5x/ (props changed)
lucene/dev/branches/branch_5x/solr/ (props changed)
lucene/dev/branches/branch_5x/solr/CHANGES.txt (contents, props changed)
lucene/dev/branches/branch_5x/solr/core/ (props changed)
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/cloud/ElectionContext.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/cloud/OverseerCollectionProcessor.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/cloud/SyncStrategy.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/cloud/ZkController.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/core/CoreContainer.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/core/CoreDescriptor.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/core/SolrCore.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/core/SolrCores.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/core/ZkContainer.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/update/DefaultSolrCoreState.java
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/update/PeerSync.java
lucene/dev/branches/branch_5x/solr/core/src/test-files/log4j.properties
lucene/dev/branches/branch_5x/solr/server/ (props changed)
lucene/dev/branches/branch_5x/solr/server/resources/log4j.properties
lucene/dev/branches/branch_5x/solr/solrj/ (props changed)
lucene/dev/branches/branch_5x/solr/solrj/src/java/org/apache/solr/common/util/ExecutorUtil.java
lucene/dev/branches/branch_5x/solr/solrj/src/test-files/log4j.properties
lucene/dev/branches/branch_5x/solr/test-framework/ (props changed)
lucene/dev/branches/branch_5x/solr/test-framework/src/java/org/apache/solr/SolrTestCaseJ4.java
lucene/dev/branches/branch_5x/solr/test-framework/src/java/org/apache/solr/cloud/AbstractFullDistribZkTestBase.java
Modified: lucene/dev/branches/branch_5x/solr/CHANGES.txt
URL:
http://svn.apache.org/viewvc/lucene/dev/branches/branch_5x/solr/CHANGES.txt?rev=1682031r1=1682030r2=1682031view=diff
==
--- lucene/dev/branches/branch_5x/solr/CHANGES.txt (original)
+++ lucene/dev/branches/branch_5x/solr/CHANGES.txt Wed May 27 13:58:30 2015
@@ -45,6 +45,8 @@ Other Changes
* SOLR-7146: MiniSolrCloudCluster based tests can fail with
ZooKeeperException NoNode for /live_nodes.
(Vamsee Yarlagadda via shalin)
+* SOLR-7590: Finish and improve MDC context logging support. (Mark Miller)
+
== 5.2.0 ==
Consult the LUCENE_CHANGES.txt file for additional, low level, changes in
this release
Modified:
[jira] [Commented] (SOLR-7590) Finish and improve MDC logging support.
[ https://issues.apache.org/jira/browse/SOLR-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561131#comment-14561131 ] Mark Miller commented on SOLR-7590: --- Looks like 5x is failing because the Supplier code is Java 8. Still trying to figure out how this appears to work on my dev machine even when running with a java 7 jvm... Finish and improve MDC logging support. --- Key: SOLR-7590 URL: https://issues.apache.org/jira/browse/SOLR-7590 Project: Solr Issue Type: Improvement Reporter: Mark Miller Assignee: Mark Miller Fix For: Trunk, 5.3 Attachments: SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch, SOLR-7950.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[ https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561044#comment-14561044 ] ASF subversion and git services commented on LUCENE-6481: - Commit 1682036 from [~mikemccand] in branch 'dev/branches/LUCENE-6481' [ https://svn.apache.org/r1682036 ] LUCENE-6481: restrict random test cases to 'smallish' bbox; switch static factory for polygon query to ctor Improve GeoPointField type to only visit high precision boundary terms --- Key: LUCENE-6481 URL: https://issues.apache.org/jira/browse/LUCENE-6481 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Nicholas Knize Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch Current GeoPointField [LUCENE-6450 | https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges along the space-filling curve that represent a provided bounding box. This determines which terms to visit in the terms dictionary and which to skip. This is suboptimal for large bounding boxes as we may end up visiting all terms (which could be quite large). This incremental improvement is to improve GeoPointField to only visit high precision terms in boundary ranges and use the postings list for ranges that are completely within the target bounding box. A separate improvement is to switch over to auto-prefix and build an Automaton representing the bounding box. That can be tracked in a separate issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-6504) implement norms with random access API
[ https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561095#comment-14561095 ] Uwe Schindler edited comment on LUCENE-6504 at 5/27/15 2:51 PM: bq. Its not unrelated, i did extensive benchmarking. That optimization is a trap, allowing too many low level implementations (3) once the index gets large. OK. That's fine. I never liked that offset=0 specialization, i think back at that time we felt it might be a good idea. I don't know the issue number anymore. You also removed the negative check in the new Multi implementation; I think this was done because of performance, too, right? I am not happy with that, because it no longer throws Exception if you try to access stuff off-slice. Maybe we can add an assert instead? was (Author: thetaphi): bq. Its not unrelated, i did extensive benchmarking. That optimization is a trap, allowing too many low level implementations (3) once the index gets large. OK. That's fine. I never liked that offset=0 specialization, i think back at that time we felt it might be a good idea. I don't know the issue number anymore. You also removed the negative check in the new Multi implementation, but I think this was done because of performance, too. I am not happy with that, because it no longer throws Exception if you try to access stuff off-slice. Maybe we can add an assert instead? implement norms with random access API -- Key: LUCENE-6504 URL: https://issues.apache.org/jira/browse/LUCENE-6504 Project: Lucene - Core Issue Type: Improvement Reporter: Robert Muir Attachments: LUCENE-6504.patch We added this api in LUCENE-5729 but we never explored implementing norms with it. These are generally the largest consumer of heap memory and often a real hassle for users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6504) implement norms with random access API
[ https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561100#comment-14561100 ] Uwe Schindler commented on LUCENE-6504: --- Sorry, the assert is there. Cancel my comment! implement norms with random access API -- Key: LUCENE-6504 URL: https://issues.apache.org/jira/browse/LUCENE-6504 Project: Lucene - Core Issue Type: Improvement Reporter: Robert Muir Attachments: LUCENE-6504.patch We added this api in LUCENE-5729 but we never explored implementing norms with it. These are generally the largest consumer of heap memory and often a real hassle for users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6504) implement norms with random access API
[
https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561040#comment-14561040
]
Robert Muir commented on LUCENE-6504:
-
{quote}
Before/now, AFAIK norms take 1 byte per field per doc of heap. I looked over
the patch briefly; does this essentially put norms off-heap?
{quote}
In the worst case. Currently they are compressed with bitpacking and other
tricks to try to be reasonable. But what was missing all along was a random
access api in Directory so that this can just be MappedByteBuffer.get(long)
(see linked issue and justification). If you want them to be in heap memory,
use fileswitchdirectory and ramdirectory.
{quote}
Does this patch also bring in accurate norms or is something else required to
enable actually utilize that?
{quote}
You have 1 byte norms because your chosen similarity squashes to that, but the
interface between similarity and indexwriter is long since lucene 4 and all
codecs test and support that.
implement norms with random access API
--
Key: LUCENE-6504
URL: https://issues.apache.org/jira/browse/LUCENE-6504
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
Attachments: LUCENE-6504.patch
We added this api in LUCENE-5729 but we never explored implementing norms
with it. These are generally the largest consumer of heap memory and often a
real hassle for users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Solr-Artifacts-5.x - Build # 844 - Failure
Build: https://builds.apache.org/job/Solr-Artifacts-5.x/844/
No tests ran.
Build Log:
[...truncated 12816 lines...]
[javac] Compiling 807 source files to
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/build/solr-core/classes/java
[javac]
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java:26:
error: package java.util.function does not exist
[javac] import java.util.function.Supplier;
[javac] ^
[javac]
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java:42:
error: cannot find symbol
[javac] private static ThreadLocalInteger CALL_DEPTH =
ThreadLocal.withInitial(new SupplierInteger() {
[javac]
^
[javac] symbol: class Supplier
[javac] location: class MDCLoggingContext
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 2 errors
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/build.xml:510: The
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/common-build.xml:389:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/lucene/common-build.xml:523:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/lucene/common-build.xml:1945:
Compile failed; see the compiler error output for details.
Total time: 1 minute 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Solr-Artifacts-5.x #843
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 38101191 bytes
Compression is 0.0%
Took 1 min 12 sec
Publishing Javadoc
Email was triggered for: Failure
Sending email for trigger: Failure
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5386) Solr hangs on spellcheck.maxCollationTries
[ https://issues.apache.org/jira/browse/SOLR-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561035#comment-14561035 ] Bob Hastings commented on SOLR-5386: I am having the same problem with Solr 4.6.0. The lock is occurring in SolrCore.getSearcher(). For me at startup both the [coreLoadExecutor-3-thread-1] and [searcherExecutor-4-thread-1] are blocked on the searhcerLock.wait() call line 1546 for SolrCore for release 4.6.0. My guess is the synchronized block needs review. I'll to more checking and let you know what I find. BTW this happens every time this line is added to the /select RequestHandler: str name=spellcheck.maxCollationTries5/str Whenever the value is greater than zero. Solr hangs on spellcheck.maxCollationTries -- Key: SOLR-5386 URL: https://issues.apache.org/jira/browse/SOLR-5386 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 4.4, 4.5 Reporter: Jeroen Steggink Labels: collate, maxCollationTries, spellcheck Attachments: threaddump.log When spellcheck.maxCollationTries is set (0) Solr hangs in combination with that requestHandler set to default=true. When I make another requestHandler default, one without the maxCollationTries, all requestHandlers work just fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6504) implement norms with random access API
[
https://issues.apache.org/jira/browse/LUCENE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561043#comment-14561043
]
Robert Muir commented on LUCENE-6504:
-
{quote}
There are some unrelated changes in ByteBufferIndexInput: You removed the
offset=0 implementation. Did you find out that this optimization for offset=0
brings no additional performance?
{quote}
Its not unrelated, i did extensive benchmarking. That optimization is a trap,
allowing too many low level implementations (3) once the index gets large.
implement norms with random access API
--
Key: LUCENE-6504
URL: https://issues.apache.org/jira/browse/LUCENE-6504
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
Attachments: LUCENE-6504.patch
We added this api in LUCENE-5729 but we never explored implementing norms
with it. These are generally the largest consumer of heap memory and often a
real hassle for users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7600) Many query functions will cast their operands in float and loose precision in the process.
Daniel Shane created SOLR-7600: -- Summary: Many query functions will cast their operands in float and loose precision in the process. Key: SOLR-7600 URL: https://issues.apache.org/jira/browse/SOLR-7600 Project: Solr Issue Type: Bug Components: Server Affects Versions: 4.10.3 Reporter: Daniel Shane Priority: Minor Function Queries like the 'max' function will cast its result to a float value even if the source value cannot be converted to floating point without loosing precision. For example, the max() function will convert dates to floats, and in the process we loose some precision (milliseconds). This is problematic if we want to sort afterwards since we do not have a millisecond precision anymore. I do not know if there is a work around short of creating a new set of query functions that would take longs / dates / etc... and return the corresponding type and name them 'long_max(), date_max() etc...' I believe it would be more intuitive if functions like max() would return the same type as what they got in their argument (assuming they are all of the same type). max(date, date) should return a date max(long, long) should return a long -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Solr-Artifacts-5.x - Build # 844 - Failure
Hmm...this is odd. Fails on my local jenkins machine as well but not on my
dev machine (using Java 7 JVM).
- Mark
On Wed, May 27, 2015 at 11:01 AM Apache Jenkins Server
jenk...@builds.apache.org wrote:
Build: https://builds.apache.org/job/Solr-Artifacts-5.x/844/
No tests ran.
Build Log:
[...truncated 12816 lines...]
[javac] Compiling 807 source files to
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/build/solr-core/classes/java
[javac]
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java:26:
error: package java.util.function does not exist
[javac] import java.util.function.Supplier;
[javac] ^
[javac]
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java:42:
error: cannot find symbol
[javac] private static ThreadLocalInteger CALL_DEPTH =
ThreadLocal.withInitial(new SupplierInteger() {
[javac]
^
[javac] symbol: class Supplier
[javac] location: class MDCLoggingContext
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 2 errors
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/build.xml:510:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/solr/common-build.xml:389:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/lucene/common-build.xml:523:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Solr-Artifacts-5.x/lucene/common-build.xml:1945:
Compile failed; see the compiler error output for details.
Total time: 1 minute 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Solr-Artifacts-5.x #843
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 38101191 bytes
Compression is 0.0%
Took 1 min 12 sec
Publishing Javadoc
Email was triggered for: Failure
Sending email for trigger: Failure
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6481) Improve GeoPointField type to only visit high precision boundary terms
[
https://issues.apache.org/jira/browse/LUCENE-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561052#comment-14561052
]
Michael McCandless commented on LUCENE-6481:
I fixed the randomized test case to only test a smallish global bbox (up to
+/- 1.5 in lat and lon directions)... when testing the full space the test runs
very very slowly, even testRandomTiny, because it can require 100s of K terms
... I'm not quite sure why (?) but the query classes do advertise that they
should be used on smallish rectangles.
Also, the test fails because of boundary cases:
{noformat}
T0: iter=57 id=6819 docID=6723 lat=-81.12143028547105 lon=-168.98618510239544
(bbox: lat=-81.22948018485512 TO -80.9313958433031 lon=-168.98618505380117 TO
-168.77958782241174) expected true but got: false deleted?=false
query=GeoPointInBBoxQuery: field=geoField: Lower Left:
[-168.98618505380117,-81.22948018485512] Upper Right:
[-168.77958782241174,-80.9313958433031]
máj 27, 2015 2:16:52 PM
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
WARNING: Uncaught exception in thread: Thread[T0,5,TGRP-TestGeoPointQuery]
java.lang.AssertionError: wrong result
at __randomizedtesting.SeedInfo.seed([4B5245DED09AE592]:0)
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.lucene.search.TestGeoPointQuery$1._run(TestGeoPointQuery.java:380)
at
org.apache.lucene.search.TestGeoPointQuery$1.run(TestGeoPointQuery.java:279)
{noformat}
It's odd because in GeoUtils#bboxContains, we do try to take the quantization
into account ...
Improve GeoPointField type to only visit high precision boundary terms
---
Key: LUCENE-6481
URL: https://issues.apache.org/jira/browse/LUCENE-6481
Project: Lucene - Core
Issue Type: Improvement
Components: core/index
Reporter: Nicholas Knize
Attachments: LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch,
LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481.patch, LUCENE-6481_WIP.patch
Current GeoPointField [LUCENE-6450 |
https://issues.apache.org/jira/browse/LUCENE-6450] computes a set of ranges
along the space-filling curve that represent a provided bounding box. This
determines which terms to visit in the terms dictionary and which to skip.
This is suboptimal for large bounding boxes as we may end up visiting all
terms (which could be quite large).
This incremental improvement is to improve GeoPointField to only visit high
precision terms in boundary ranges and use the postings list for ranges that
are completely within the target bounding box.
A separate improvement is to switch over to auto-prefix and build an
Automaton representing the bounding box. That can be tracked in a separate
issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7115) UpdateLog can miss closing transaction log objects.
[ https://issues.apache.org/jira/browse/SOLR-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561238#comment-14561238 ] Mark Miller commented on SOLR-7115: --- It's possible this was caused by SOLR-7478. UpdateLog can miss closing transaction log objects. --- Key: SOLR-7115 URL: https://issues.apache.org/jira/browse/SOLR-7115 Project: Solr Issue Type: Bug Reporter: Mark Miller I've seen this happen on YourKit and in various tests - especially since adding resource release tracking to the log objects. Now I've got a test that catches it in SOLR-7113. It seems that in precommit, if prevTlog is not null, we need to close it because we are going to overwrite prevTlog with a new log. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7601) If a test fails, that error should be reported and not an error about resources that were not closed later.
[ https://issues.apache.org/jira/browse/SOLR-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561331#comment-14561331 ] Dawid Weiss commented on SOLR-7601: --- This should be close. AfterClass is invoked inside the class rule that detects failures so certain exceptions could happen in between, but individual tests should taint that suiteFailureMarker. You will not be able to react to hanging threads -- this is initiated and executed outside of the test classes, followed by interrupt on the test thread (which then may throw random crap at the fan). This is chicken-and-egg problem, I don't think it's quite solvable. You should scan for failures in the log, top-to-bottom... If a test fails, that error should be reported and not an error about resources that were not closed later. --- Key: SOLR-7601 URL: https://issues.apache.org/jira/browse/SOLR-7601 Project: Solr Issue Type: Test Reporter: Mark Miller Assignee: Mark Miller Fix For: Trunk, 5.3 Attachments: SOLR-7601.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Solr Ref Guide 5.2 RC Planning - aiming for RC0 on 2015-06-01
I'm volunteering to be the RM for the 5.2 ref guide. Now that anshum has started on the 5.2 (code) RC process, we need to knuckle down on the ref guide as well. I'm about to start reviewing summarizing the CHANGES.txt list for notable things that need doc'ed which i'll post on the TODO page -- but in the meantime if you've worked on a new Solr feature in 5.2, please make sure you've updated the corrisponding page in the ref guide -- you shouldn't be waiting until the last minute to do that, this part of the process is just about looking for things that have fallen through the cracks. My goal is to cut the first ref guide RC no later then monday (2015-06-01) and -- as usual -- cancel any ref guide vote that is ongoing if/when a new code RC is published tha affects anything in the ref guide. -Hoss http://www.lucidworks.com/ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Hmm, bug?
Problem here is that you changed the schema without completely re-indexing. In fact, when changes like this are made I typically blow away the entire data directory or delete/recreate the collection. At the Lucene layer, there's no schema and certainly no going back to update already closed segments because a schema changed. So things get confused when trying to sort since multiple segments have different definitions for the same field. The loose coupling between the schema and what's already in the index from an older schema definition is one of the things we learn to deal with. In general, when doing much of anything to the schema except _adding_ fields, I recommend starting over completely. You can do this crudely by stopping all your servers and deleting the data directory (as in 'rm -rf data) then restarting. Or, just use the collections API to delete then re-add the collection. Best, Erick On Wed, May 27, 2015 at 9:21 AM, Gus Heck gus.h...@gmail.com wrote: So I did the following (seemingly) reasonable set of steps... Set up a Solr Cloud instance with mutable managed schema (data driven config) Tossed a couple docs in just to see it work. Tried to sort on 'title', realized title had been guessed as multivalued. Changed the field to type=string (as opposed to strings) using schema api Tried to sort on title The result is trace: java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for field 'title' (expected=SORTED). Use UninvertingReader or index with docvalues.\n\tat org.apache.lucene.index.DocValues.checkField(DocValues.java:208) Resending the curl command to re-index the docs hangs However, I have not played with schemaless or the schema api before, so maybe I've missed something, and my steps are not reasonable, and thus this sanity check before I file a bug. -Gus -- http://www.the111shift.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7601) If a test fails, that error should be reported and not an error about resources that were not closed later.
[ https://issues.apache.org/jira/browse/SOLR-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561280#comment-14561280 ] Mark Miller commented on SOLR-7601: --- Hmm...that is not quite what I'm after. Another attempt coming. Trying to solve test failures where jenkins reports things like searchers not being closed, but the real issue is something else (like hanging threads or something else). If a test fails, that error should be reported and not an error about resources that were not closed later. --- Key: SOLR-7601 URL: https://issues.apache.org/jira/browse/SOLR-7601 Project: Solr Issue Type: Test Reporter: Mark Miller Assignee: Mark Miller Fix For: Trunk, 5.3 Attachments: SOLR-7601.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7601) If a test fails, that error should be reported and not an error about resources that were not closed later.
Mark Miller created SOLR-7601: - Summary: If a test fails, that error should be reported and not an error about resources that were not closed later. Key: SOLR-7601 URL: https://issues.apache.org/jira/browse/SOLR-7601 Project: Solr Issue Type: Test Reporter: Mark Miller Assignee: Mark Miller Fix For: Trunk, 5.3 Attachments: SOLR-7601.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7601) If a test fails, that error should be reported and not an error about resources that were not closed later.
[ https://issues.apache.org/jira/browse/SOLR-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-7601: -- Attachment: SOLR-7601.patch If a test fails, that error should be reported and not an error about resources that were not closed later. --- Key: SOLR-7601 URL: https://issues.apache.org/jira/browse/SOLR-7601 Project: Solr Issue Type: Test Reporter: Mark Miller Assignee: Mark Miller Fix For: Trunk, 5.3 Attachments: SOLR-7601.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Hmm, bug (schemaless/schema api)?
Ok, thanks. Sorry for the uninformative subject line. I realized after the fact it was not very good (ok really bad :) ). What you say is what I had generally understood to be the case in the past with regular static schemas, but I had hoped something newer and more flexible was implied with the advent of an API, and the schemaless configuration. Although there is a caveat at the top of the wiki page, statements like When modifying the schema with the REST API, a core reload will automatically occur in order for the changes to be available immediately. had given me hope. Guess you haven't got that far after all. -Gus On Wed, May 27, 2015 at 1:03 PM, Erick Erickson erickerick...@gmail.com wrote: Problem here is that you changed the schema without completely re-indexing. In fact, when changes like this are made I typically blow away the entire data directory or delete/recreate the collection. At the Lucene layer, there's no schema and certainly no going back to update already closed segments because a schema changed. So things get confused when trying to sort since multiple segments have different definitions for the same field. The loose coupling between the schema and what's already in the index from an older schema definition is one of the things we learn to deal with. In general, when doing much of anything to the schema except _adding_ fields, I recommend starting over completely. You can do this crudely by stopping all your servers and deleting the data directory (as in 'rm -rf data) then restarting. Or, just use the collections API to delete then re-add the collection. Best, Erick On Wed, May 27, 2015 at 9:21 AM, Gus Heck gus.h...@gmail.com wrote: So I did the following (seemingly) reasonable set of steps... Set up a Solr Cloud instance with mutable managed schema (data driven config) Tossed a couple docs in just to see it work. Tried to sort on 'title', realized title had been guessed as multivalued. Changed the field to type=string (as opposed to strings) using schema api Tried to sort on title The result is trace: java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for field 'title' (expected=SORTED). Use UninvertingReader or index with docvalues.\n\tat org.apache.lucene.index.DocValues.checkField(DocValues.java:208) Resending the curl command to re-index the docs hangs However, I have not played with schemaless or the schema api before, so maybe I've missed something, and my steps are not reasonable, and thus this sanity check before I file a bug. -Gus -- http://www.the111shift.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- http://www.the111shift.com
[jira] [Commented] (SOLR-7118) ChaosMonkeyNothingIsSafeTest fails with too many update fails
[
https://issues.apache.org/jira/browse/SOLR-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561241#comment-14561241
]
Mark Miller commented on SOLR-7118:
---
Saw another errant fail on jenkins cluster - much, much rarer at least - going
to raise again.
ChaosMonkeyNothingIsSafeTest fails with too many update fails
-
Key: SOLR-7118
URL: https://issues.apache.org/jira/browse/SOLR-7118
Project: Solr
Issue Type: Bug
Components: SolrCloud, Tests
Affects Versions: 5.0
Reporter: Shalin Shekhar Mangar
Fix For: Trunk, 5.2
Attachments: SOLR-7118.patch
There are frequent failures on both trunk and branch_5x with the following
message:
{code}
java.lang.AssertionError: There were too many update fails - we expect it can
happen, but shouldn't easily
at
__randomizedtesting.SeedInfo.seed([786DB0FD42626C16:F98B3EE5353D0C2A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:224)
at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:878)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7570) Config APIs should not modify the ConfigSet
[
https://issues.apache.org/jira/browse/SOLR-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated SOLR-7570:
Attachment: SOLR-7570.patch
Here's the first cut of an idea.
* Adds a writeResource(String, byte[], Location) method to SolrResourceLoader
(Location can be LOCAL, COLLECTION, CONFIGSET)
* SolrResourceLoader takes both a local instance dir and a shared config dir.
These can be the same if you're not using a configset.
* The standard resource loader looks in three places for resources:
** core instance dir
** configset
** classpath
* The ZK resource loader looks in four places:
** core instance dir
** collection-specific config
** zk config
** classpath
You can write to either the local core instance dir, or to the
collection-specific config (I added CONFIG as a location in case we want to use
that later for things like specifying where a particular resource was found,
but that can be taken out if it's not adding anything now).
Writing to the collection-specific config uses version-tracking to implement
optimistic concurrency.
There are tests for the standard resource loader and the ZK resource loader.
This is still pretty rough around the edges, and I haven't run the full test
suite or started cutting over existing code to using the new API, but it's a
start. What do people think?
Config APIs should not modify the ConfigSet
---
Key: SOLR-7570
URL: https://issues.apache.org/jira/browse/SOLR-7570
Project: Solr
Issue Type: Improvement
Reporter: Tomás Fernández Löbbe
Attachments: SOLR-7570.patch
Originally discussed here:
http://mail-archives.apache.org/mod_mbox/lucene-dev/201505.mbox/%3CCAMJgJxSXCHxDzJs5-C-pKFDEBQD6JbgxB=-xp7u143ekmgp...@mail.gmail.com%3E
The ConfigSet used to create a collection should be read-only. Changes made
via any of the Config APIs should only be applied to the collection where the
operation is done and no to other collections that may be using the same
ConfigSet. As discussed in the dev list:
When a collection is created we should have two things, an immutable part
(the ConfigSet) and a mutable part (configoverlay, generated schema, etc).
The ConfigSet will still be placed in ZooKeeper under /configs but the
mutable part should be placed under /collections/$COLLECTION_NAME/…
[~romseygeek] suggested:
{quote}
A nice way of doing it would be to make it part of the SolrResourceLoader
interface. The ZK resource loader could check in the collection-specific
zknode first, and then under configs/, and we could add a writeResource()
method that writes to the collection-specific node as well. Then all config
I/O goes via the resource loader, and we have a way of keeping certain parts
immutable.
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7344) Use two thread pools, one for internal requests and one for external, to avoid distributed deadlock and decrease the number of threads that need to be created.
[ https://issues.apache.org/jira/browse/SOLR-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561627#comment-14561627 ] Yonik Seeley commented on SOLR-7344: bq. Note even for this approach, we need to implement a way to identify an internal request Shard sub-requests will always have isShard=true param set. bq. The idea here is to implement a custom Servlet filter similar to Jetty QoSFilter Looks interesting. I wonder what the performance implications of using continuations are? Use two thread pools, one for internal requests and one for external, to avoid distributed deadlock and decrease the number of threads that need to be created. --- Key: SOLR-7344 URL: https://issues.apache.org/jira/browse/SOLR-7344 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Mark Miller Attachments: SOLR-7344.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7602) Frequent MultiThreadedOCPTest failures on Jenkins
Anshum Gupta created SOLR-7602:
--
Summary: Frequent MultiThreadedOCPTest failures on Jenkins
Key: SOLR-7602
URL: https://issues.apache.org/jira/browse/SOLR-7602
Project: Solr
Issue Type: Bug
Reporter: Anshum Gupta
The number of failed MultiThreadedOCPTest runs on Jenkins has gone up
drastically since Apr 30, 2015.
{code}
REGRESSION: org.apache.solr.cloud.MultiThreadedOCPTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=6313,
name=parallelCoreAdminExecutor-1988-thread-15, state=RUNNABLE,
group=TGRP-MultiThreadedOCPTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=6313,
name=parallelCoreAdminExecutor-1988-thread-15, state=RUNNABLE,
group=TGRP-MultiThreadedOCPTest]
at
__randomizedtesting.SeedInfo.seed([1FD11A82D96D185B:97852558779175A3]:0)
Caused by: java.lang.AssertionError: Too many closes on SolrCore
at __randomizedtesting.SeedInfo.seed([1FD11A82D96D185B]:0)
at org.apache.solr.core.SolrCore.close(SolrCore.java:1138)
at org.apache.solr.common.util.IOUtils.closeQuietly(IOUtils.java:31)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:535)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:494)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:598)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)
at
org.apache.solr.handler.admin.CoreAdminHandler$ParallelCoreAdminHandlerThread.run(CoreAdminHandler.java:1219)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
Last failure:
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/12665/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6485) Add a custom separator break iterator
[ https://issues.apache.org/jira/browse/LUCENE-6485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-6485. - Resolution: Fixed Fix Version/s: 5.3 Trunk Thanks Luca! This is a nice alternative to WholeBreakIterator. Add a custom separator break iterator - Key: LUCENE-6485 URL: https://issues.apache.org/jira/browse/LUCENE-6485 Project: Lucene - Core Issue Type: Improvement Reporter: Luca Cavanna Fix For: Trunk, 5.3 Attachments: LUCENE-6485.patch Lucene currently includes a WholeBreakIterator used to highlight entire fields using the postings highlighter, without breaking their content into sentences. I would like to contribute a CustomSeparatorBreakIterator that breaks when a custom char separator is found in the text. This can be used for instance when wanting to highlight entire fields, value per value. One can subclass PostingsHighlighter and have getMultiValueSeparator return a control character, like U+ , then use the custom break iterator to break on U+ so that one snippet per value will be generated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7555) Display total space and available space in Admin
[ https://issues.apache.org/jira/browse/SOLR-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561575#comment-14561575 ] Eric Pugh commented on SOLR-7555: - Humm... You know, I saw that RAMDirectory implemented a interface called Accountable that was for tracking memory usage.So we could modify the HdfsDirectory and FSDirectory to implement that. Then in SystemInfoHandler we do a instanceof check, and then pull that data? [~ehatcher] [~mariusneo] what do you guys think? Display total space and available space in Admin Key: SOLR-7555 URL: https://issues.apache.org/jira/browse/SOLR-7555 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 5.1 Reporter: Eric Pugh Assignee: Erik Hatcher Priority: Minor Fix For: 5.2 Attachments: SOLR-7555-display_disk_space.patch, SOLR-7555-display_disk_space_v2.patch, SOLR-7555.patch Frequently I have access to the Solr Admin console, but not the underlying server, and I'm curious how much space remains available. This little patch exposes total Volume size as well as the usable space remaining: !https://monosnap.com/file/VqlReekCFwpK6utI3lP18fbPqrGI4b.png! I'm not sure if this is the best place to put this, as every shard will share the same data, so maybe it should be on the top level Dashboard? Also not sure what to call the fields! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: svn commit: r1682057 - /lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/logging/MDCLoggingContext.java
Hi Mark,
I am wondering why you be so verbose in Java 8's code. The main reason for
ThreadLocal.withInitial() is to use it like that:
private static final ThreadLocalInteger CALL_DEPTH =
ThreadLocal.withInitial(() - 0);
Using a Supplier without a lambda is not the intention behind this method /API
:-)
In Java 7, you can use the code like this fix comitted. If you want it verbose
in Java 8, I would also use the Java 7 code...
Uwe
-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de
-Original Message-
From: markrmil...@apache.org [mailto:markrmil...@apache.org]
Sent: Wednesday, May 27, 2015 5:28 PM
To: comm...@lucene.apache.org
Subject: svn commit: r1682057 -
/lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/loggin
g/MDCLoggingContext.java
Author: markrmiller
Date: Wed May 27 15:27:43 2015
New Revision: 1682057
URL: http://svn.apache.org/r1682057
Log:
SOLR-7590: Java8 code - Java7 for branch_5x.
Modified:
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/logging
/MDCLoggingContext.java
Modified:
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/logging
/MDCLoggingContext.java
URL:
http://svn.apache.org/viewvc/lucene/dev/branches/branch_5x/solr/core/sr
c/java/org/apache/solr/logging/MDCLoggingContext.java?rev=1682057r1=
1682056r2=1682057view=diff
==
---
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/logging
/MDCLoggingContext.java (original)
+++
lucene/dev/branches/branch_5x/solr/core/src/java/org/apache/solr/log
+++ ging/MDCLoggingContext.java Wed May 27 15:27:43 2015
@@ -23,7 +23,6 @@ import static org.apache.solr.common.clo import static
org.apache.solr.common.cloud.ZkStateReader.REPLICA_PROP;
import static
org.apache.solr.common.cloud.ZkStateReader.SHARD_ID_PROP;
-import java.util.function.Supplier;
import org.apache.solr.cloud.CloudDescriptor;
import org.apache.solr.cloud.ZkController;
@@ -39,12 +38,13 @@ import org.slf4j.MDC;
*/
public class MDCLoggingContext {
// When a thread sets context and finds that the context is already set, we
should noop and ignore the finally clear
- private static ThreadLocalInteger CALL_DEPTH =
ThreadLocal.withInitial(new SupplierInteger() {
+ private static ThreadLocalInteger CALL_DEPTH = new
+ ThreadLocalInteger() {
@Override
-public Integer get() {
+protected Integer initialValue() {
return 0;
}
- });
+ };
+
private static void setCollection(String collection) {
if (collection != null) {
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6459) [suggest] Query Interface for suggest API
[
https://issues.apache.org/jira/browse/LUCENE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561616#comment-14561616
]
Michael McCandless commented on LUCENE-6459:
Thanks [~areek], new patch looks great, passes {{ant precommit}} ... I'll
commit soon.
[suggest] Query Interface for suggest API
-
Key: LUCENE-6459
URL: https://issues.apache.org/jira/browse/LUCENE-6459
Project: Lucene - Core
Issue Type: New Feature
Components: core/search
Affects Versions: 5.1
Reporter: Areek Zillur
Assignee: Areek Zillur
Fix For: Trunk, 5.x, 5.1
Attachments: LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch,
LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch,
LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch, LUCENE-6459.patch
This patch factors out common indexing/search API used by the recently
introduced [NRTSuggester|https://issues.apache.org/jira/browse/LUCENE-6339].
The motivation is to provide a query interface for FST-based fields
(*SuggestField* and *ContextSuggestField*)
to enable suggestion scoring and more powerful automaton queries.
Previously, only prefix ‘queries’ with index-time weights were supported but
we can also support:
* Prefix queries expressed as regular expressions: get suggestions that
match multiple prefixes
** *Example:* _star\[wa\|tr\]_ matches _starwars_ and _startrek_
* Fuzzy Prefix queries supporting scoring: get typo tolerant suggestions
scored by how close they are to the query prefix
** *Example:* querying for _seper_ will score _separate_ higher then
_superstitious_
* Context Queries: get suggestions boosted and/or filtered based on their
indexed contexts (meta data)
** *Boost example:* get typo tolerant suggestions on song names with
prefix _like a roling_ boosting songs with
genre _rock_ and _indie_
** *Filter example:* get suggestion on all file names starting with
_finan_ only for _user1_ and _user2_
h3. Suggest API
{code}
SuggestIndexSearcher searcher = new SuggestIndexSearcher(reader);
CompletionQuery query = ...
TopSuggestDocs suggest = searcher.suggest(query, num);
{code}
h3. CompletionQuery
*CompletionQuery* is used to query *SuggestField* and *ContextSuggestField*.
A *CompletionQuery* produces a *CompletionWeight*,
which allows *CompletionQuery* implementations to pass in an automaton that
will be intersected with a FST and allows boosting and
meta data extraction from the intersected partial paths. A *CompletionWeight*
produces a *CompletionScorer*. A *CompletionScorer*
executes a Top N search against the FST with the provided automaton, scoring
and filtering all matched paths.
h4. PrefixCompletionQuery
Return documents with values that match the prefix of an analyzed term text
Documents are sorted according to their suggest field weight.
{code}
PrefixCompletionQuery(Analyzer analyzer, Term term)
{code}
h4. RegexCompletionQuery
Return documents with values that match the prefix of a regular expression
Documents are sorted according to their suggest field weight.
{code}
RegexCompletionQuery(Term term)
{code}
h4. FuzzyCompletionQuery
Return documents with values that has prefixes within a specified edit
distance of an analyzed term text.
Documents are ‘boosted’ by the number of matching prefix letters of the
suggestion with respect to the original term text.
{code}
FuzzyCompletionQuery(Analyzer analyzer, Term term)
{code}
h5. Scoring
{{suggestion_weight * boost}}
where {{suggestion_weight}} and {{boost}} are all integers.
{{boost = # of prefix characters matched}}
h4. ContextQuery
Return documents that match a {{CompletionQuery}} filtered and/or boosted by
provided context(s).
{code}
ContextQuery(CompletionQuery query)
contextQuery.addContext(CharSequence context, int boost, boolean exact)
{code}
*NOTE:* {{ContextQuery}} should be used with {{ContextSuggestField}} to query
suggestions boosted and/or filtered by contexts.
Running {{ContextQuery}} against a {{SuggestField}} will error out.
h5. Scoring
{{suggestion_weight * context_boost}}
where {{suggestion_weight}} and {{context_boost}} are all integers
When used with {{FuzzyCompletionQuery}},
{{suggestion_weight * (context_boost + fuzzy_boost)}}
h3. Context Suggest Field
To use {{ContextQuery}}, use {{ContextSuggestField}} instead of
{{SuggestField}}. Any {{CompletionQuery}} can be used with
{{ContextSuggestField}}, the default behaviour is to return suggestions from
*all* contexts. {{Context}} for every completion hit
can be accessed through {{SuggestScoreDoc#context}}.
{code}
ContextSuggestField(String name, CollectionCharSequence contexts, String
value, int weight)
{code}
--
This
[VOTE] 5.2.0 RC1
Please vote for the first release candidate for Lucene/Solr 5.2.0 The artifacts can be downloaded from: https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.2.0-RC1-rev1682085 You can run the smoke tester directly with this command: python3 -u dev-tools/scripts/smokeTestRelease.py https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.2.0-RC1-rev1682085 I've run the test suite a few times, the smoke tester, basic collection creation, startup, indexing, and query. Here's my +1. SUCCESS! [0:31:23.943785] P.S: I hit failure in MultiThreadedOCPTest 2 times while creating the RC, so I'm looking at what's triggering it in parallel to make sure that we're not overlooking a problem. As it has been failing on Jenkins frequently, I've created SOLR-7602 https://issues.apache.org/jira/browse/SOLR-7602to track this. -- Anshum Gupta
[jira] [Commented] (LUCENE-6485) Add a custom separator break iterator
[ https://issues.apache.org/jira/browse/LUCENE-6485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561622#comment-14561622 ] ASF subversion and git services commented on LUCENE-6485: - Commit 1682114 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1682114 ] LUCENE-6485: Add CustomSeparatorBreakIterator Add a custom separator break iterator - Key: LUCENE-6485 URL: https://issues.apache.org/jira/browse/LUCENE-6485 Project: Lucene - Core Issue Type: Improvement Reporter: Luca Cavanna Attachments: LUCENE-6485.patch Lucene currently includes a WholeBreakIterator used to highlight entire fields using the postings highlighter, without breaking their content into sentences. I would like to contribute a CustomSeparatorBreakIterator that breaks when a custom char separator is found in the text. This can be used for instance when wanting to highlight entire fields, value per value. One can subclass PostingsHighlighter and have getMultiValueSeparator return a control character, like U+ , then use the custom break iterator to break on U+ so that one snippet per value will be generated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6485) Add a custom separator break iterator
[ https://issues.apache.org/jira/browse/LUCENE-6485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561638#comment-14561638 ] ASF subversion and git services commented on LUCENE-6485: - Commit 1682115 from [~rcmuir] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1682115 ] LUCENE-6485: Add CustomSeparatorBreakIterator Add a custom separator break iterator - Key: LUCENE-6485 URL: https://issues.apache.org/jira/browse/LUCENE-6485 Project: Lucene - Core Issue Type: Improvement Reporter: Luca Cavanna Attachments: LUCENE-6485.patch Lucene currently includes a WholeBreakIterator used to highlight entire fields using the postings highlighter, without breaking their content into sentences. I would like to contribute a CustomSeparatorBreakIterator that breaks when a custom char separator is found in the text. This can be used for instance when wanting to highlight entire fields, value per value. One can subclass PostingsHighlighter and have getMultiValueSeparator return a control character, like U+ , then use the custom break iterator to break on U+ so that one snippet per value will be generated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
