RE: reuseAddress default in Solr jetty.xml
I agree, sigkill is typically the last resort.. On 3 Mar 2015 00:49, Reitzel, Charles charles.reit...@tiaa-cref.org wrote: My bad. Too long away from sockets since cleaning up those shutdown handlers. Your point is well taken, on the server side the risks of consuming a stray echo packet are fairly low (but non-zero, if you’ve ever spent any quality time with tcpdump/wireshark). Still, in a production setting, SIGKILL (aka “kill -9”) should be a last resort after more reasonable methods (e.g. SIGINT, SIGTERM, SIGSTOP) have failed. *From:* Ramkumar R. Aiyengar [mailto:andyetitmo...@gmail.com] *Sent:* Monday, March 02, 2015 7:00 PM *To:* dev@lucene.apache.org *Subject:* RE: reuseAddress default in Solr jetty.xml No, reuseAddress doesn't allow you to have two processes, old and new, listen to the same port. There's no option which allows you to do that. Tl;DR This can happen when you have a connection to a server which gets killed hard and comes back up immediately So here's what happens. When a server normally shuts down, it triggers an active close on all open TCP connections it has. That sends a three way msg exchange with the remote recipient (FIN, FIN+ACK, ACK) at the end of which the socket is closed and the kernel puts it in a TIME_WAIT state for a few minutes in the background (depends on the OS, maximum tends to be 4 mins). This is needed to allow for reordered older packets to reach the machine just in case. Now typically if the server restarts within that period and tries to bind again to the same port, the kernel is smart enough to not complain that there is an existing socket in TIME_WAIT, because it knows the last sequence number it used for the final message in the previous process, and since sequence numbers are always increasing, it can reject any messages before that sequence number as a new process has now taken the port. Trouble is with abnormal shutdown. There's no time for a proper goodbye, so the kernel marks the socket to respond to remote packets with a rude RST (reset). Since there has been no goodbye with the remote end, it also doesn't know the last sequence number to delineate if a new process binds to the same port. Hence by default it denies binding to the new port for the TIME_WAIT period to avoid the off chance a stray packet gets picked up by the new process and utterly confuses it. By setting reuseAddress, you are essentially waiving off this protection. Note that this possibility of confusion is unbelievably miniscule in the first place (both the source and destination host:port should be the same and the client port is generally randomly allocated). If the port we are talking of is a local port, it's almost impossible -- you have bigger problems if a TCP packet is lost or delayed within the same machine! As to Shawn's point, for Solr's stop port, you essentially need to be trying to actively shutdown the server using the stop port, or be within a few minutes of such an attempt while the server is killed. Just the server being killed without any active connection to it is not going to cause this issue. Hi Ram, It appears the problem is that the old solr/jetty process is actually still running when the new solr/jetty process is started. That’s the problem that needs fixing. This is not a rare problem in systems with worker threads dedicated to different tasks. These threads need to wake up in response to the shutdown signal/command, as well the normal inputs. It’s a bug I’ve created and fixed a couple times over the years … :-)I wouldn’t know where to start with Solr. But, as I say, re-using the port is a band-aid. I’ve yet to see a case where it is the best solution. best, Charlie *From:* Ramkumar R. Aiyengar [mailto:andyetitmo...@gmail.com] *Sent:* Saturday, February 28, 2015 8:15 PM *To:* dev@lucene.apache.org *Subject:* Re: reuseAddress default in Solr jetty.xml Hey Charles, see my explanation above on why this is needed. If Solr has to be killed, it would generally be immediately restarted. This would normally not the case, except when things are potentially misconfigured or if there is a bug, but not doing so makes the impact worse.. In any case, turns out really that reuseAddress is true by default for the connectors we use, so that really isn't the issue. The issue more specifically is that the stop port doesn't do it, so the actual port by itself starts just fine on a restart, but the stop port fails to bind -- and there's no way currently in Jetty to configure that. Based on my question in the jetty mailing list, I have now created an issue for them.. https://bugs.eclipse.org/bugs/show_bug.cgi?id=461133 On Fri, Feb 27, 2015 at 3:03 PM, Reitzel, Charles charles.reit...@tiaa-cref.org wrote: Disclaimer: I’m not a Solr committer. But, as a developer, I’ve never seen a good case for reusing the listening port. Better to find and fix
Re: Welcome Ramkumar Aiyengar as Lucene/Solr committer
Welcome On Mar 3, 2015 8:57 AM, Noble Paul noble.p...@gmail.com wrote: Awesome , It was long overdue welcome ramkumar On Tue, Mar 3, 2015 at 12:39 AM, Tomás Fernández Löbbe tomasflo...@gmail.com wrote: Welcome Ramkumar! On Mon, Mar 2, 2015 at 8:01 AM, Ryan Ernst r...@iernst.net wrote: Welcome! On Mar 1, 2015 8:40 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: I'm pleased to announce that Ramkumar Aiyengar has accepted the PMC's invitation to become a committer. Ramkumar, it's tradition that you introduce yourself with a brief bio. Your handle andyetitmoves has already added to the “lucene LDAP group, so you now have commit privileges. Please test this by adding yourself to the committers section of the Who We Are page on the website: http://lucene.apache.org/whoweare.html (use the ASF CMS bookmarklet at the bottom of the page here: https://cms.apache.org/#bookmark - more info here http://www.apache.org/dev/cms.html). The ASF dev page also has lots of useful links: http://www.apache.org/dev/. Congratulations and welcome! -- Regards, Shalin Shekhar Mangar. -- - Noble Paul
Re: DocValues instead of stored values
On Tue, 2015-03-03 at 06:03 +0100, Shalin Shekhar Mangar wrote: Also, the docs for field function query say that it is for numerics but it works great with strings too. It probably won't work with multi-valued (I haven't tried them). I just tested with Solr 4.8. It supports field(dv_field) for single valued Strings and performance is great, relative to our requirements. Unfortunately asking for multi-value gives me the error can not use FieldCache on multivalued field just as you suspected. Tracking through the Solr 5.0 code, the field parser returns a ValueSource, providing FunctionValues that, unfortunately for us, are limited to single-value. We'll have to extract the multi-values explicitly with faceting or export, as Joel suggests, for the time being. - Toke Eskildsen, State and University Library, Denmark - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2707 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2707/ 5 tests failed. FAILED: org.apache.solr.cloud.HttpPartitionTest.test Error Message: org.apache.http.NoHttpResponseException: The target server failed to respond Stack Trace: org.apache.solr.client.solrj.SolrServerException: org.apache.http.NoHttpResponseException: The target server failed to respond at __randomizedtesting.SeedInfo.seed([4A7A2182F28A5B66:C22E1E585C76369E]:0) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:899) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:752) at org.apache.solr.cloud.HttpPartitionTest.doSendDoc(HttpPartitionTest.java:484) at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:501) at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:193) at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:950) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:925) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at
[jira] [Commented] (LUCENE-6299) IndexWriter's enforcement of 2.1B doc limits is buggy
[ https://issues.apache.org/jira/browse/LUCENE-6299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344830#comment-14344830 ] Michael McCandless commented on LUCENE-6299: Argh: the CHANGES.txt entry for this issue went into the wrong place in 4.10.x, even though the fix was in fact committed for 4.10.4 IndexWriter's enforcement of 2.1B doc limits is buggy - Key: LUCENE-6299 URL: https://issues.apache.org/jira/browse/LUCENE-6299 Project: Lucene - Core Issue Type: Bug Reporter: Michael McCandless Assignee: Michael McCandless Priority: Blocker Fix For: 4.10.4, 5.0, Trunk, 5.1 Attachments: LUCENE-6299-410x.patch, LUCENE-6299.patch, LUCENE-6299.patch, LUCENE-6299.patch, LUCENE-6299.patch, LUCENE-6299_addIndexes.patch E.g. if you pass an already 2.1B docs to either addIndexes, it can fail to enforce properly. IW's private reserveDocs should refuse to accept negative values. IW.deleteAll fails to set the pendingNumDocs to 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6321) Make oal.index.Term more defensive
[
https://issues.apache.org/jira/browse/LUCENE-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345005#comment-14345005
]
Robert Muir commented on LUCENE-6321:
-
Looks ok in general.
Not sure about Term.clone() changes, i think maybe this should be an explicit
deepCopyOf method just like bytesref?
In general, who is modifying term bytes? This seems like an abuse case, and its
gonna be problematic everywhere in lucene apis if you do this.
But maybe its a good idea to do a little extra clone work to make these core
queries defensive. Another thing we could do instead would be a best effort
check where we actually throw concurrentmodificationexception or something,
that might be more interesting.
Make oal.index.Term more defensive
--
Key: LUCENE-6321
URL: https://issues.apache.org/jira/browse/LUCENE-6321
Project: Lucene - Core
Issue Type: Bug
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Attachments: LUCENE-6321.patch
oal.index.Term has a Term(String field, BytesRef termBytes) constructor. Even
though it warns that the term bytes should not be reused, I'm wondering that
we should make it more defensive.
{noformat}
* pWARNING: the provided BytesRef is not copied, but used directly.
* Therefore the bytes should not be modified after construction, for
* example, you should clone a copy by {@link BytesRef#deepCopyOf}
* rather than pass reused bytes from a TermsEnum.
{noformat}
For example if you have term queries in your query cache and they are
modified in-place, it would have very bad consequences and would be hard to
diagnose.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Security release because of Jetty Security issue: #JetLeak
Hi, due to the security leak in the Jetty webserver we should think about updating the Solr releases: As a Lucene 4.10.4 release is in the RC phase, we should better delay it and check if the bundled Jetty is vulnerable. http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html Here is a testing scipt to check our release: https://github.com/GDSSecurity/Jetleak-Testing-Script Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Artifacts-5.x - Build # 780 - Failure
Build: https://builds.apache.org/job/Lucene-Artifacts-5.x/780/ No tests ran. Build Log: [...truncated 12495 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Artifacts-5.x/lucene/build.xml:389: java.net.ConnectException: Operation timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.init(HttpClient.java:211) at sun.net.www.http.HttpClient.New(HttpClient.java:308) at sun.net.www.http.HttpClient.New(HttpClient.java:326) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850) at org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660) at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579) at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569) Total time: 7 minutes 38 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Sending artifact delta relative to Lucene-Artifacts-5.x #779 Archived 13 artifacts Archive block size is 32768 Received 1078 blocks and 140836903 bytes Compression is 20.1% Took 2 min 39 sec Publishing Javadoc Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 775 - Still Failing
I'm looking into it. On Tue, Mar 3, 2015 at 3:52 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/775/ 1 tests failed. FAILED: org.apache.lucene.search.TestLRUQueryCache.testRandom Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([8AFB9C1E9D5CF8AF:F8B7B9112C3C4EDC]:0) at org.apache.lucene.search.TermScorer.score(TermScorer.java:70) at org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:46) at org.apache.lucene.search.DisjunctionScorer.score(DisjunctionScorer.java:219) at org.apache.lucene.search.ReqExclScorer.score(ReqExclScorer.java:121) at org.apache.lucene.search.BooleanScorer$OrCollector.collect(BooleanScorer.java:142) at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:181) at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:171) at org.apache.lucene.search.BooleanScorer$BulkScorerAndDoc.score(BooleanScorer.java:60) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:210) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:264) at org.apache.lucene.search.BooleanScorer.score(BooleanScorer.java:289) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:79) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:63) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:694) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:102) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617) at org.apache.lucene.search.IndexSearcher.count(IndexSearcher.java:290) at org.apache.lucene.search.TestLRUQueryCache.testRandom(TestLRUQueryCache.java:923) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Updated] (SOLR-7173) Fix ReplicationFactorTest on Windows
[ https://issues.apache.org/jira/browse/SOLR-7173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated SOLR-7173: --- Attachment: SOLR-7173.patch Thanks for your suggestion. I've updated the patch; I've added the warning before retrying. However, instead of the recursive approach (through my initial few attempts, couldn't get the tests to pass), I went for an iterative one; do you suggest I go with the recursive approach? Also, since I had copied the initial retry logic from HttpPartitionTest's sendDoc(), should I update that code as well with this iterative/recursive retry? Fix ReplicationFactorTest on Windows Key: SOLR-7173 URL: https://issues.apache.org/jira/browse/SOLR-7173 Project: Solr Issue Type: Bug Reporter: Ishan Chattopadhyaya Fix For: 5.1 Attachments: SOLR-7173.patch, SOLR-7173.patch The ReplicationFactorTest fails on the Windows build with NoHttpResponseException, as seen here: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4502/testReport/junit/org.apache.solr.cloud/ReplicationFactorTest/test/ Adding a retry logic similar to HttpPartitionTest's doSend() method makes the test pass on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5332) Add preserve original setting to the EdgeNGramFilterFactory
[ https://issues.apache.org/jira/browse/SOLR-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345260#comment-14345260 ] Furkan KAMACI commented on SOLR-5332: - [~simon.endele] You can check my patch at SOLR-5152. I've applied a patch there and this issue become a duplicate. Add preserve original setting to the EdgeNGramFilterFactory - Key: SOLR-5332 URL: https://issues.apache.org/jira/browse/SOLR-5332 Project: Solr Issue Type: Wish Affects Versions: 4.4, 4.5, 4.5.1, 4.6 Reporter: Alexander S. Fix For: 5.1 Hi, as described here: http://lucene.472066.n3.nabble.com/Help-to-figure-out-why-query-does-not-match-td4086967.html the problem is in that if you have these 2 strings to index: 1. facebook.com/someuser.1 2. facebook.com/someveryandverylongusername and the edge ngram filter factory with min and max gram size settings 2 and 25, search requests for these urls will fail. But search requests for: 1. facebook.com/someuser 2. facebook.com/someveryandverylonguserna will work properly. It's because first url has 1 at the end, which is lover than the allowed min gram size. In the second url the user name is longer than the max gram size (27 characters). Would be good to have a preserve original option, that will add the original string to the index if it does not fit the allowed gram size, so that 1 and someveryandverylongusername tokens will also be added to the index. Best, Alex -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7184) Update Jetty to 9.2.9
[ https://issues.apache.org/jira/browse/SOLR-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated SOLR-7184: Attachment: SOLR-7184.patch Patch. Will commit in a moment. Update Jetty to 9.2.9 - Key: SOLR-7184 URL: https://issues.apache.org/jira/browse/SOLR-7184 Project: Solr Issue Type: Task Components: Server Affects Versions: Trunk Reporter: Uwe Schindler Priority: Blocker Fix For: Trunk Attachments: SOLR-7184.patch Due to the security leak in the Jetty webserver have to update the branches that use Jetty 9.2.x to latest version 9.2.9: - http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html - https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7184) Update Jetty to 9.2.9
[ https://issues.apache.org/jira/browse/SOLR-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345299#comment-14345299 ] ASF subversion and git services commented on SOLR-7184: --- Commit 1663736 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1663736 ] SOLR-7184: Update Jetty Update Jetty to 9.2.9 - Key: SOLR-7184 URL: https://issues.apache.org/jira/browse/SOLR-7184 Project: Solr Issue Type: Task Components: Server Affects Versions: Trunk Reporter: Uwe Schindler Priority: Blocker Fix For: Trunk Attachments: SOLR-7184.patch Due to the security leak in the Jetty webserver have to update the branches that use Jetty 9.2.x to latest version 9.2.9: - http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html - https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 775 - Still Failing
one option is to wrap with constantscorer somewhere and still keep the optimization? On Tue, Mar 3, 2015 at 11:32 AM, Adrien Grand jpou...@gmail.com wrote: OK, I found the reason. We have an optimization in disjunctions to stop calling matches() after a single matching clause has been found if scores are not needed, but it does not play well with the fact that some scorers always call score() on the underlying clauses even if scores are not needed (BooleanScorer in this case, maybe there are other ones). So I'll disable the optimization. We'll need to better document when it is legal to call score() if we want to bring it back. On Tue, Mar 3, 2015 at 3:57 PM, Adrien Grand jpou...@gmail.com wrote: I'm looking into it. On Tue, Mar 3, 2015 at 3:52 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/775/ 1 tests failed. FAILED: org.apache.lucene.search.TestLRUQueryCache.testRandom Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([8AFB9C1E9D5CF8AF:F8B7B9112C3C4EDC]:0) at org.apache.lucene.search.TermScorer.score(TermScorer.java:70) at org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:46) at org.apache.lucene.search.DisjunctionScorer.score(DisjunctionScorer.java:219) at org.apache.lucene.search.ReqExclScorer.score(ReqExclScorer.java:121) at org.apache.lucene.search.BooleanScorer$OrCollector.collect(BooleanScorer.java:142) at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:181) at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:171) at org.apache.lucene.search.BooleanScorer$BulkScorerAndDoc.score(BooleanScorer.java:60) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:210) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:264) at org.apache.lucene.search.BooleanScorer.score(BooleanScorer.java:289) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:79) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:63) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:694) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:102) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617) at org.apache.lucene.search.IndexSearcher.count(IndexSearcher.java:290) at org.apache.lucene.search.TestLRUQueryCache.testRandom(TestLRUQueryCache.java:923) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at
RE: Security release because of Jetty Security issue: #JetLeak
Hi, Here ist he official statement: https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md So we are fine in Solr 5.0 and 4.9.x, but we need to update our checkout to at least latest Jetty 9.2.9 version. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Tuesday, March 03, 2015 3:04 PM To: dev@lucene.apache.org Subject: RE: Security release because of Jetty Security issue: #JetLeak It looks like Jetty 7 and Jetty 8 are not affected, only Jetty 9. So I think we are safe :-) Investigating... Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Tuesday, March 03, 2015 1:44 PM To: dev@lucene.apache.org Subject: Security release because of Jetty Security issue: #JetLeak Hi, due to the security leak in the Jetty webserver we should think about updating the Solr releases: As a Lucene 4.10.4 release is in the RC phase, we should better delay it and check if the bundled Jetty is vulnerable. http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remot e- leakage-of-shared-buffers-in-je.html Here is a testing scipt to check our release: https://github.com/GDSSecurity/Jetleak-Testing-Script Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-5.x-Windows (32bit/jdk1.7.0_76) - Build # 4520 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/4412/ Java: 32bit/jdk1.7.0_76 -server -XX:+UseG1GC 1 tests failed. FAILED: org.apache.solr.cloud.ReplicationFactorTest.test Error Message: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:61337/repfacttest_c8n_1x3_shard1_replica3 Stack Trace: org.apache.solr.client.solrj.SolrServerException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:61337/repfacttest_c8n_1x3_shard1_replica3 at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:597) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:918) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:809) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:752) at org.apache.solr.cloud.ReplicationFactorTest.testRf3(ReplicationFactorTest.java:283) at org.apache.solr.cloud.ReplicationFactorTest.test(ReplicationFactorTest.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:950) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:925) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at
[jira] [Resolved] (SOLR-7184) Update Jetty to 9.2.9
[ https://issues.apache.org/jira/browse/SOLR-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler resolved SOLR-7184. - Resolution: Fixed Assignee: Uwe Schindler I also fixed CHANGES.txt in branch_5x (incorrect Jetty versions for 5.0 and 5.1!!!) Update Jetty to 9.2.9 - Key: SOLR-7184 URL: https://issues.apache.org/jira/browse/SOLR-7184 Project: Solr Issue Type: Task Components: Server Affects Versions: Trunk Reporter: Uwe Schindler Assignee: Uwe Schindler Priority: Blocker Fix For: Trunk Attachments: SOLR-7184.patch Due to the security leak in the Jetty webserver have to update the branches that use Jetty 9.2.x to latest version 9.2.9: - http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html - https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7182) Make the Schema-API a first class citizen of SolrJ
[ https://issues.apache.org/jira/browse/SOLR-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345304#comment-14345304 ] Noble Paul commented on SOLR-7182: -- I would say , don't worry about any particular release. Just build it . all the point releases will happen in quick succession . If it does not go in 5.1 , there will be a 5.2 pretty soon Make the Schema-API a first class citizen of SolrJ -- Key: SOLR-7182 URL: https://issues.apache.org/jira/browse/SOLR-7182 Project: Solr Issue Type: Improvement Components: SolrJ Affects Versions: 5.0 Environment: any Reporter: Sven Windisch Priority: Minor Labels: api, schema, solrj Fix For: 5.1 There are several Solr APIs that are handled as first class citizens in SolrJ, esp. the Node API and the Collections API, i.e. they have their own xxxAdminRequest Object extended from the SolrRequest Class. As someone who programmatically changes Schemas a lot, I had hoped to see the Schema API handled first class in release 5.0, too. As far as I dug into the code and docs of SolrJ 5.0, that did not happen. If there is a reasonable point why this won't happen at all, I would really like to hear it. If the only reason is, that nobody had time for this, I would happily help out here. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Commiter review request: SOLR-7174
I shall do it, Alexandre
On Tue, Mar 3, 2015 at 8:24 PM, Alexandre Rafalovitch arafa...@gmail.com
wrote:
https://issues.apache.org/jira/browse/SOLR-7174
DIH TikaEntityProcessor does not work as an inner entity as it does
not reinit correctly between outer loop iterations.
Reported against 5.0, trivial fix patch provided, tests pass,
reporter's problem is solved.
Just need somebody with commit access to review that the solution
('init' method override) is acceptable and merge it into trunk, 5.0
and possibly into 4.10.x.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
--
-
Noble Paul
[jira] [Updated] (SOLR-7073) Add an API to add a jar to a collection's classpath
[
https://issues.apache.org/jira/browse/SOLR-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-7073:
-
Attachment: SOLR-7073.patch
more tests and cleanup
Add an API to add a jar to a collection's classpath
---
Key: SOLR-7073
URL: https://issues.apache.org/jira/browse/SOLR-7073
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Attachments: SOLR-7073.patch, SOLR-7073.patch, SOLR-7073.patch
The idea of having separate classloaders for each component may be counter
productive. This aims to add a jar dependency to the collection itself and
each core belonging to that collection will have the jar in the classpath
As we load everything from the .system collection , we cannot make the core
loading delayed till .system is fully loaded and is available .
There is a new set of config commands to manage the dependencies on a
collection level. It is possible to have more than one jar as a dependency.
The lib attribute is same as the blob name that we use in the blob store API
example:
{code}
curl http://localhost:8983/solr/collection1/config -H
'Content-type:application/json' -d '{
add-runtimelib : {name: jarname , version:2 },
update-runtimelib :{name: jarname ,version:3},
delete-runtimelib :jarname
}'
{code}
The same is added to the overlay.json .
The default SolrResourceLoader does not have visibility to these jars .
There is a classloader that can access these jars which is made available
only to those components which are specially annotated
Every pluggable component can have an optional extra attribute called
{{runtimeLib=true}}, which means, these components are not be loaded at core
load time. They are tried to be loaded on demand and if all the dependency
jars are not available at the component load time an error is thrown .
example of registering a valueSourceParser which depends on the runtime
classloader
{code}
curl http://localhost:8983/solr/collection1/config -H
'Content-type:application/json' -d '{
create-valuesourceparser : {name: nvl ,
runtimeLib: true,
class:solr.org.apache.solr.search.function.NvlValueSourceParser ,
nvlFloatValue:0.0 }
}'
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7184) Update Jetty to 9.2.9
Uwe Schindler created SOLR-7184: --- Summary: Update Jetty to 9.2.9 Key: SOLR-7184 URL: https://issues.apache.org/jira/browse/SOLR-7184 Project: Solr Issue Type: Task Reporter: Uwe Schindler Priority: Blocker Due to the security leak in the Jetty webserver have to update the branches that use Jetty 9.2.x to latest version 9.2.9: - http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html - https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7184) Update Jetty to 9.2.9
[ https://issues.apache.org/jira/browse/SOLR-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated SOLR-7184: Affects Version/s: Trunk Fix Version/s: Trunk branch_5x is also not affected, still on Jetty 8. Update Jetty to 9.2.9 - Key: SOLR-7184 URL: https://issues.apache.org/jira/browse/SOLR-7184 Project: Solr Issue Type: Task Affects Versions: Trunk Reporter: Uwe Schindler Priority: Blocker Fix For: Trunk Due to the security leak in the Jetty webserver have to update the branches that use Jetty 9.2.x to latest version 9.2.9: - http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html - https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7184) Update Jetty to 9.2.9
[ https://issues.apache.org/jira/browse/SOLR-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated SOLR-7184: Component/s: Server Update Jetty to 9.2.9 - Key: SOLR-7184 URL: https://issues.apache.org/jira/browse/SOLR-7184 Project: Solr Issue Type: Task Components: Server Affects Versions: Trunk Reporter: Uwe Schindler Priority: Blocker Fix For: Trunk Due to the security leak in the Jetty webserver have to update the branches that use Jetty 9.2.x to latest version 9.2.9: - http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html - https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 775 - Still Failing
OK, I found the reason. We have an optimization in disjunctions to stop calling matches() after a single matching clause has been found if scores are not needed, but it does not play well with the fact that some scorers always call score() on the underlying clauses even if scores are not needed (BooleanScorer in this case, maybe there are other ones). So I'll disable the optimization. We'll need to better document when it is legal to call score() if we want to bring it back. On Tue, Mar 3, 2015 at 3:57 PM, Adrien Grand jpou...@gmail.com wrote: I'm looking into it. On Tue, Mar 3, 2015 at 3:52 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/775/ 1 tests failed. FAILED: org.apache.lucene.search.TestLRUQueryCache.testRandom Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([8AFB9C1E9D5CF8AF:F8B7B9112C3C4EDC]:0) at org.apache.lucene.search.TermScorer.score(TermScorer.java:70) at org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:46) at org.apache.lucene.search.DisjunctionScorer.score(DisjunctionScorer.java:219) at org.apache.lucene.search.ReqExclScorer.score(ReqExclScorer.java:121) at org.apache.lucene.search.BooleanScorer$OrCollector.collect(BooleanScorer.java:142) at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:181) at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:171) at org.apache.lucene.search.BooleanScorer$BulkScorerAndDoc.score(BooleanScorer.java:60) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:210) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:264) at org.apache.lucene.search.BooleanScorer.score(BooleanScorer.java:289) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:79) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:63) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:694) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:102) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617) at org.apache.lucene.search.IndexSearcher.count(IndexSearcher.java:290) at org.apache.lucene.search.TestLRUQueryCache.testRandom(TestLRUQueryCache.java:923) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at
[jira] [Commented] (SOLR-7184) Update Jetty to 9.2.9
[ https://issues.apache.org/jira/browse/SOLR-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345307#comment-14345307 ] Uwe Schindler commented on SOLR-7184: - *Is there a reason why branch_5x is not yet on Jetty 9?* Update Jetty to 9.2.9 - Key: SOLR-7184 URL: https://issues.apache.org/jira/browse/SOLR-7184 Project: Solr Issue Type: Task Components: Server Affects Versions: Trunk Reporter: Uwe Schindler Assignee: Uwe Schindler Priority: Blocker Fix For: Trunk Attachments: SOLR-7184.patch Due to the security leak in the Jetty webserver have to update the branches that use Jetty 9.2.x to latest version 9.2.9: - http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html - https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7176) allow zkcli to modify JSON
[
https://issues.apache.org/jira/browse/SOLR-7176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345323#comment-14345323
]
Mark Miller commented on SOLR-7176:
---
bq. There is already a clusterprop command why do we need a zkCli command
There is already a response to this question above ?
allow zkcli to modify JSON
--
Key: SOLR-7176
URL: https://issues.apache.org/jira/browse/SOLR-7176
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Priority: Minor
To enable SSL, we have instructions like the following:
{code}
server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd put
/clusterprops.json '{urlScheme:https}'
{code}
Overwriting the value won't work well when we have more properties to put in
clusterprops. We should be able to change individual values or perhaps merge
values.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6328) Make Filter.clone and Filter.setBoost throw an UnsupportedOperationException
[ https://issues.apache.org/jira/browse/LUCENE-6328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand updated LUCENE-6328: - Attachment: LUCENE-6328.patch Here is a patch that makes filters rewrite to a CSQ around themselves and throw an UnsupportedOperationException in setBoost() and clone(). For cloning, maybe another option would be to make clone() abstract in Filter so that existing filters would fail to compile unless implemented instead of relying on the default impl in Query? Make Filter.clone and Filter.setBoost throw an UnsupportedOperationException Key: LUCENE-6328 URL: https://issues.apache.org/jira/browse/LUCENE-6328 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: Trunk, 5.1 Attachments: LUCENE-6328.patch The rewrite process uses a combination of calls to clone() and setBoost(boost) in order to rewrite queries. This is a bit weird for filters given that they were not originally designed to care about scoring. Using a filter directly as a query fails unit tests today since filters do not pass the QueryUtils checks: it is expected that cloning and changing the boost results in an instance which is unequal. However existing filters do not take into account the getBoost() parameter inherited from Query so this test fails. I think it would be less error-prone to throw an UnsupportedOperationException for clone() and setBoost() on filters and disable the check in QueryUtils for filters. In order to keep rewriting working, filters could rewrite to a CSQ around themselves so that clone() and setBoost() would be called on the CSQ. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2708 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2708/ 5 tests failed. FAILED: org.apache.solr.cloud.HttpPartitionTest.test Error Message: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:41105/wqyo/c8n_1x2_shard1_replica1 Stack Trace: org.apache.solr.client.solrj.SolrServerException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:41105/wqyo/c8n_1x2_shard1_replica1 at __randomizedtesting.SeedInfo.seed([2E07612BB1432F0:8AB449C815E85F08]:0) at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:597) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:918) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:809) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:752) at org.apache.solr.cloud.HttpPartitionTest.doSendDoc(HttpPartitionTest.java:484) at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:501) at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:193) at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:950) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:925) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (LUCENE-1518) Merge Query and Filter classes
[ https://issues.apache.org/jira/browse/LUCENE-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345052#comment-14345052 ] Adrien Grand commented on LUCENE-1518: -- I opened LUCENE-6328 to discuss options for this problem... Merge Query and Filter classes -- Key: LUCENE-1518 URL: https://issues.apache.org/jira/browse/LUCENE-1518 Project: Lucene - Core Issue Type: Improvement Components: core/search Affects Versions: 2.4 Reporter: Uwe Schindler Fix For: Trunk, 5.1 Attachments: LUCENE-1518.patch, LUCENE-1518.patch This issue presents a patch, that merges Queries and Filters in a way, that the new Filter class extends Query. This would make it possible, to use every filter as a query. The new abstract filter class would contain all methods of ConstantScoreQuery, deprecate ConstantScoreQuery. If somebody implements the Filter's getDocIdSet()/bits() methods he has nothing more to do, he could just use the filter as a normal query. I do not want to completely convert Filters to ConstantScoreQueries. The idea is to combine Queries and Filters in such a way, that every Filter can automatically be used at all places where a Query can be used (e.g. also alone a search query without any other constraint). For that, the abstract Query methods must be implemented and return a default weight for Filters which is the current ConstantScore Logic. If the filter is used as a real filter (where the API wants a Filter), the getDocIdSet part could be directly used, the weight is useless (as it is currently, too). The constant score default implementation is only used when the Filter is used as a Query (e.g. as direct parameter to Searcher.search()). For the special case of BooleanQueries combining Filters and Queries the idea is, to optimize the BooleanQuery logic in such a way, that it detects if a BooleanClause is a Filter (using instanceof) and then directly uses the Filter API and not take the burden of the ConstantScoreQuery (see LUCENE-1345). Here some ideas how to implement Searcher.search() with Query and Filter: - User runs Searcher.search() using a Filter as the only parameter. As every Filter is also a ConstantScoreQuery, the query can be executed and returns score 1.0 for all matching documents. - User runs Searcher.search() using a Query as the only parameter: No change, all is the same as before - User runs Searcher.search() using a BooleanQuery as parameter: If the BooleanQuery does not contain a Query that is subclass of Filter (the new Filter) everything as usual. If the BooleanQuery only contains exactly one Filter and nothing else the Filter is used as a constant score query. If BooleanQuery contains clauses with Queries and Filters the new algorithm could be used: The queries are executed and the results filtered with the filters. For the user this has the main advantage: That he can construct his query using a simplified API without thinking about Filters oder Queries, you can just combine clauses together. The scorer/weight logic then identifies the cases to use the filter or the query weight API. Just like the query optimizer of a RDB. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6328) Make Filter.clone and Filter.setBoost throw an UnsupportedOperationException
Adrien Grand created LUCENE-6328: Summary: Make Filter.clone and Filter.setBoost throw an UnsupportedOperationException Key: LUCENE-6328 URL: https://issues.apache.org/jira/browse/LUCENE-6328 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: Trunk, 5.1 The rewrite process uses a combination of calls to clone() and setBoost(boost) in order to rewrite queries. This is a bit weird for filters given that they were not originally designed to care about scoring. Using a filter directly as a query fails unit tests today since filters do not pass the QueryUtils checks: it is expected that cloning and changing the boost results in an instance which is unequal. However existing filters do not take into account the getBoost() parameter inherited from Query so this test fails. I think it would be less error-prone to throw an UnsupportedOperationException for clone() and setBoost() on filters and disable the check in QueryUtils for filters. In order to keep rewriting working, filters could rewrite to a CSQ around themselves so that clone() and setBoost() would be called on the CSQ. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.10-Linux (32bit/jdk1.9.0-ea-b47) - Build # 11749 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.10-Linux/52/ Java: 32bit/jdk1.9.0-ea-b47 -server -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.lucene.index.TestBackwardsCompatibility.testAllVersionsTested Error Message: Saw Version constant but no corresponding back-compat index: 4104.cfs Stack Trace: java.lang.AssertionError: Saw Version constant but no corresponding back-compat index: 4104.cfs at __randomizedtesting.SeedInfo.seed([9AD4E24BA3C2C62A:8A0B0D0B1755D006]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.lucene.index.TestBackwardsCompatibility.testAllVersionsTested(TestBackwardsCompatibility.java:454) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 1194 lines...] [junit4] Suite: org.apache.lucene.index.TestBackwardsCompatibility [junit4] 2 NOTE: reproduce with: ant test -Dtestcase=TestBackwardsCompatibility -Dtests.method=testAllVersionsTested -Dtests.seed=9AD4E24BA3C2C62A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fr_LU
[jira] [Commented] (LUCENE-6321) Make oal.index.Term more defensive
[
https://issues.apache.org/jira/browse/LUCENE-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345045#comment-14345045
]
Adrien Grand commented on LUCENE-6321:
--
OK to not make Term cloneable.
The test that showed the issue is ThreadedIndexingAndSearchingTestCase which
has the following code:
{noformat}
while (System.currentTimeMillis() stopTimeMS) {
BytesRef term = termsEnum.next();
if (term == null) {
totTermCount.set(seenTermCount);
break;
}
seenTermCount++;
// search 30 terms
if ((seenTermCount + shift) % trigger == 0) {
//if (VERBOSE) {
//System.out.println(Thread.currentThread().getName() +
now search body: + term.utf8ToString());
//}
totHits.addAndGet(runQuery(s, new TermQuery(new
Term(body, term;
}
}
{noformat}
Make oal.index.Term more defensive
--
Key: LUCENE-6321
URL: https://issues.apache.org/jira/browse/LUCENE-6321
Project: Lucene - Core
Issue Type: Bug
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Attachments: LUCENE-6321.patch
oal.index.Term has a Term(String field, BytesRef termBytes) constructor. Even
though it warns that the term bytes should not be reused, I'm wondering that
we should make it more defensive.
{noformat}
* pWARNING: the provided BytesRef is not copied, but used directly.
* Therefore the bytes should not be modified after construction, for
* example, you should clone a copy by {@link BytesRef#deepCopyOf}
* rather than pass reused bytes from a TermsEnum.
{noformat}
For example if you have term queries in your query cache and they are
modified in-place, it would have very bad consequences and would be hard to
diagnose.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6321) Make oal.index.Term more defensive
[
https://issues.apache.org/jira/browse/LUCENE-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345048#comment-14345048
]
Adrien Grand commented on LUCENE-6321:
--
bq. Another thing we could do instead would be a best effort check where we
actually throw concurrentmodificationexception or something, that might be more
interesting.
Right, I started working on that and it is actually quite easy to implement.
The only thing I'm worried about is that it would only appear on the first time
that you need an eviction. So if your cache has a capacity of eg. 10,000
queries, it would take time before you notice the issue. And once it happens,
all queries that insert entries into the cache would fail since no eviction
could work anymore. So we probably need to also make queries a bit more
defensive.
Make oal.index.Term more defensive
--
Key: LUCENE-6321
URL: https://issues.apache.org/jira/browse/LUCENE-6321
Project: Lucene - Core
Issue Type: Bug
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Attachments: LUCENE-6321.patch
oal.index.Term has a Term(String field, BytesRef termBytes) constructor. Even
though it warns that the term bytes should not be reused, I'm wondering that
we should make it more defensive.
{noformat}
* pWARNING: the provided BytesRef is not copied, but used directly.
* Therefore the bytes should not be modified after construction, for
* example, you should clone a copy by {@link BytesRef#deepCopyOf}
* rather than pass reused bytes from a TermsEnum.
{noformat}
For example if you have term queries in your query cache and they are
modified in-place, it would have very bad consequences and would be hard to
diagnose.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7184) Update Jetty to 9.2.9
[ https://issues.apache.org/jira/browse/SOLR-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345324#comment-14345324 ] Shalin Shekhar Mangar commented on SOLR-7184: - bq. Is there a reason why branch_5x is not yet on Jetty 9? # We were too close to the 5.0 release to port the jetty upgrade to 5.0 from trunk # There are more failures in trunk related to NoHttpResponseException in tests that use SocketProxy class on Jetty9. I haven't had the time to really figure out why. Update Jetty to 9.2.9 - Key: SOLR-7184 URL: https://issues.apache.org/jira/browse/SOLR-7184 Project: Solr Issue Type: Task Components: Server Affects Versions: Trunk Reporter: Uwe Schindler Assignee: Uwe Schindler Priority: Blocker Fix For: Trunk Attachments: SOLR-7184.patch Due to the security leak in the Jetty webserver have to update the branches that use Jetty 9.2.x to latest version 9.2.9: - http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html - https://github.com/eclipse/jetty.project/blob/master/advisories/2015-02-24-httpparser-error-buffer-bleed.md -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6304) Add MatchNoDocsQuery that matches no documents
[ https://issues.apache.org/jira/browse/LUCENE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345327#comment-14345327 ] Adrien Grand commented on LUCENE-6304: -- Feels wrong to me to override hashCode but not equals. I think we should move this class part of hashCode() to Query.hashCode()? (ie. return Float.floatToIntBits(getBoost()) ^ getClass().hashCode();) If it is controversial then I'm happy with the previous patch that overrides both equals and hashCode(). Add MatchNoDocsQuery that matches no documents -- Key: LUCENE-6304 URL: https://issues.apache.org/jira/browse/LUCENE-6304 Project: Lucene - Core Issue Type: Improvement Components: core/search Affects Versions: 5.0 Reporter: Lee Hinman Priority: Minor Attachments: LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch As a followup to LUCENE-6298, it would be nice to have an explicit MatchNoDocsQuery to indicate that no documents should be matched. This would hopefully be a better indicator than a BooleanQuery with no clauses or (even worse) null. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 775 - Still Failing
Well, we should think about the api and not rush it. i was just thinking that right now, this problem is really a disjunctions one, in that it does something wrong if you call score(), even though you said you didnt need them. but in this case, if it wrapped its subs with constantscorer, it would then be well-behaved ? On Tue, Mar 3, 2015 at 12:02 PM, Adrien Grand jpou...@gmail.com wrote: OK, it works for me too. Then I'll also add assertions that you do not call score() if you passed needsScores=false. On Tue, Mar 3, 2015 at 5:38 PM, Robert Muir rcm...@gmail.com wrote: one option is to wrap with constantscorer somewhere and still keep the optimization? On Tue, Mar 3, 2015 at 11:32 AM, Adrien Grand jpou...@gmail.com wrote: OK, I found the reason. We have an optimization in disjunctions to stop calling matches() after a single matching clause has been found if scores are not needed, but it does not play well with the fact that some scorers always call score() on the underlying clauses even if scores are not needed (BooleanScorer in this case, maybe there are other ones). So I'll disable the optimization. We'll need to better document when it is legal to call score() if we want to bring it back. On Tue, Mar 3, 2015 at 3:57 PM, Adrien Grand jpou...@gmail.com wrote: I'm looking into it. On Tue, Mar 3, 2015 at 3:52 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/775/ 1 tests failed. FAILED: org.apache.lucene.search.TestLRUQueryCache.testRandom Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([8AFB9C1E9D5CF8AF:F8B7B9112C3C4EDC]:0) at org.apache.lucene.search.TermScorer.score(TermScorer.java:70) at org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:46) at org.apache.lucene.search.DisjunctionScorer.score(DisjunctionScorer.java:219) at org.apache.lucene.search.ReqExclScorer.score(ReqExclScorer.java:121) at org.apache.lucene.search.BooleanScorer$OrCollector.collect(BooleanScorer.java:142) at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:181) at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:171) at org.apache.lucene.search.BooleanScorer$BulkScorerAndDoc.score(BooleanScorer.java:60) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:210) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:264) at org.apache.lucene.search.BooleanScorer.score(BooleanScorer.java:289) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:79) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:63) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:694) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:102) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617) at org.apache.lucene.search.IndexSearcher.count(IndexSearcher.java:290) at org.apache.lucene.search.TestLRUQueryCache.testRandom(TestLRUQueryCache.java:923) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at
[jira] [Commented] (SOLR-7177) ConcurrentUpdateSolrClient should log connection information on http failures
[
https://issues.apache.org/jira/browse/SOLR-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345347#comment-14345347
]
Mark Miller commented on SOLR-7177:
---
Just to be clear by the way, just discussing, not pushing back on the current
patch.
ConcurrentUpdateSolrClient should log connection information on http failures
--
Key: SOLR-7177
URL: https://issues.apache.org/jira/browse/SOLR-7177
Project: Solr
Issue Type: Improvement
Affects Versions: 4.10.3, 5.0
Reporter: Vamsee Yarlagadda
Priority: Minor
Attachments: SOLR-7177.patch, SOLR-7177v2.patch
I notice when there is an http connection failure, we simply log the error
but not the connection information. It would be good to log this info to make
debugging easier.
e.g:
1.
{code}
2015-02-27 08:56:51,503 ERROR org.apache.solr.update.StreamingSolrServers:
error
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:235)
{code}
2.
{code}
2015-02-27 10:26:12,363 ERROR org.apache.solr.update.StreamingSolrServers:
error
org.apache.http.NoHttpResponseException: The target server failed to respond
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:235)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
Re: Commiter review request: SOLR-7174
Thank you.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 3 March 2015 at 11:44, Noble Paul noble.p...@gmail.com wrote:
I shall do it, Alexandre
On Tue, Mar 3, 2015 at 8:24 PM, Alexandre Rafalovitch arafa...@gmail.com
wrote:
https://issues.apache.org/jira/browse/SOLR-7174
DIH TikaEntityProcessor does not work as an inner entity as it does
not reinit correctly between outer loop iterations.
Reported against 5.0, trivial fix patch provided, tests pass,
reporter's problem is solved.
Just need somebody with commit access to review that the solution
('init' method override) is acceptable and merge it into trunk, 5.0
and possibly into 4.10.x.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
--
-
Noble Paul
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7185) null pointer exception at AbstractSolrSecondaryIndex.java
MASSIMO CELLI created SOLR-7185: --- Summary: null pointer exception at AbstractSolrSecondaryIndex.java Key: SOLR-7185 URL: https://issues.apache.org/jira/browse/SOLR-7185 Project: Solr Issue Type: Bug Components: search Affects Versions: 4.6 Reporter: MASSIMO CELLI Priority: Minor a few hundreds of the following NPE were reported in the system.log: INFO [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,435 AbstractSolrSecondaryIndex.java (line 1491) Found 4 rows with expired columns. WARN [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,436 AbstractSolrSecondaryIndex.java (line 1505) java.lang.NullPointerException at com.datastax.bdp.search.solr.Cql3SolrSecondaryIndex.buildIndexedColumnFamily(Cql3SolrSecondaryIndex.java:98) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.doApplyUpdates(AbstractSolrSecondaryIndex.java:932) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.access$1300(AbstractSolrSecondaryIndex.java:94) at com .datastax.bdp.search.solr.AbstractSolrSecondaryIndex$TTLIndexRebuildTask.run(AbstractSolrSecondaryIndex.java:1501) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) WARN [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,436 AbstractSolrSecondaryIndex.java (line 1505) java.lang.NullPointerException at com.datastax.bdp.search.solr.Cql3SolrSecondaryIndex.buildIndexedColumnFamily(Cql3SolrSecondaryIndex.java:98) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.doApplyUpdates(AbstractSolrSecondaryIndex.java:932) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.access$1300(AbstractSolrSecondaryIndex.java:94) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex$TTLIndexRebuildTask.run(AbstractSolrSecondaryIndex.java:1501) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-7185) null pointer exception at AbstractSolrSecondaryIndex.java
[ https://issues.apache.org/jira/browse/SOLR-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-7185. - Resolution: Invalid Hi Massimo, that error is from the DataStax fork of Solr. Please report this bug to DataStax. I am closing this issue. null pointer exception at AbstractSolrSecondaryIndex.java -- Key: SOLR-7185 URL: https://issues.apache.org/jira/browse/SOLR-7185 Project: Solr Issue Type: Bug Components: search Affects Versions: 4.6 Reporter: MASSIMO CELLI Priority: Minor a few hundreds of the following NPE were reported in the system.log: INFO [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,435 AbstractSolrSecondaryIndex.java (line 1491) Found 4 rows with expired columns. WARN [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,436 AbstractSolrSecondaryIndex.java (line 1505) java.lang.NullPointerException at com.datastax.bdp.search.solr.Cql3SolrSecondaryIndex.buildIndexedColumnFamily(Cql3SolrSecondaryIndex.java:98) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.doApplyUpdates(AbstractSolrSecondaryIndex.java:932) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.access$1300(AbstractSolrSecondaryIndex.java:94) at com .datastax.bdp.search.solr.AbstractSolrSecondaryIndex$TTLIndexRebuildTask.run(AbstractSolrSecondaryIndex.java:1501) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) WARN [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,436 AbstractSolrSecondaryIndex.java (line 1505) java.lang.NullPointerException at com.datastax.bdp.search.solr.Cql3SolrSecondaryIndex.buildIndexedColumnFamily(Cql3SolrSecondaryIndex.java:98) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.doApplyUpdates(AbstractSolrSecondaryIndex.java:932) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.access$1300(AbstractSolrSecondaryIndex.java:94) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex$TTLIndexRebuildTask.run(AbstractSolrSecondaryIndex.java:1501) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6329) Calling score() should be ok even if needsScores is false
Adrien Grand created LUCENE-6329: Summary: Calling score() should be ok even if needsScores is false Key: LUCENE-6329 URL: https://issues.apache.org/jira/browse/LUCENE-6329 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: Trunk, 5.1 Currently if you use approximations on a disjunction scorer and call score() while needsScores was set to false, then you might run into exceptions. The reason is that when scores are not needed then the scorer doesn't maintain the list of scorers positioned on the current doc ID and you can eg. end up with exhausted scorers in this list. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7115) UpdateLog can miss closing transaction log objects.
[ https://issues.apache.org/jira/browse/SOLR-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345335#comment-14345335 ] Mark Miller commented on SOLR-7115: --- I was still seeing it fail without that when I left last week. I'll try again soon. One of the 'doc time to live' distrib tests also occasionally fails on jenkins with it. It was failing consistently for me, but I'm sure there is some timing involved. UpdateLog can miss closing transaction log objects. --- Key: SOLR-7115 URL: https://issues.apache.org/jira/browse/SOLR-7115 Project: Solr Issue Type: Bug Reporter: Mark Miller I've seen this happen on YourKit and in various tests - especially since adding resource release tracking to the log objects. Now I've got a test that catches it in SOLR-7113. It seems that in precommit, if prevTlog is not null, we need to close it because we are going to overwrite prevTlog with a new log. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 775 - Still Failing
OK, it works for me too. Then I'll also add assertions that you do not call score() if you passed needsScores=false. On Tue, Mar 3, 2015 at 5:38 PM, Robert Muir rcm...@gmail.com wrote: one option is to wrap with constantscorer somewhere and still keep the optimization? On Tue, Mar 3, 2015 at 11:32 AM, Adrien Grand jpou...@gmail.com wrote: OK, I found the reason. We have an optimization in disjunctions to stop calling matches() after a single matching clause has been found if scores are not needed, but it does not play well with the fact that some scorers always call score() on the underlying clauses even if scores are not needed (BooleanScorer in this case, maybe there are other ones). So I'll disable the optimization. We'll need to better document when it is legal to call score() if we want to bring it back. On Tue, Mar 3, 2015 at 3:57 PM, Adrien Grand jpou...@gmail.com wrote: I'm looking into it. On Tue, Mar 3, 2015 at 3:52 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/775/ 1 tests failed. FAILED: org.apache.lucene.search.TestLRUQueryCache.testRandom Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([8AFB9C1E9D5CF8AF:F8B7B9112C3C4EDC]:0) at org.apache.lucene.search.TermScorer.score(TermScorer.java:70) at org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:46) at org.apache.lucene.search.DisjunctionScorer.score(DisjunctionScorer.java:219) at org.apache.lucene.search.ReqExclScorer.score(ReqExclScorer.java:121) at org.apache.lucene.search.BooleanScorer$OrCollector.collect(BooleanScorer.java:142) at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:181) at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:171) at org.apache.lucene.search.BooleanScorer$BulkScorerAndDoc.score(BooleanScorer.java:60) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:210) at org.apache.lucene.search.BooleanScorer.scoreWindow(BooleanScorer.java:264) at org.apache.lucene.search.BooleanScorer.score(BooleanScorer.java:289) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:79) at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:63) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:694) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:102) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617) at org.apache.lucene.search.IndexSearcher.count(IndexSearcher.java:290) at org.apache.lucene.search.TestLRUQueryCache.testRandom(TestLRUQueryCache.java:923) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at
[jira] [Commented] (SOLR-7177) ConcurrentUpdateSolrClient should log connection information on http failures
[
https://issues.apache.org/jira/browse/SOLR-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345340#comment-14345340
]
Mark Miller commented on SOLR-7177:
---
bq. Adding this info at this class will also benefit all the end users who are
directly using solrj api to talk with Solr service.
In that case you know the URL - you had to init the class with it and this is
part of your client code. You can log whatever you want if the call fails on
the client side.
ConcurrentUpdateSolrClient should log connection information on http failures
--
Key: SOLR-7177
URL: https://issues.apache.org/jira/browse/SOLR-7177
Project: Solr
Issue Type: Improvement
Affects Versions: 4.10.3, 5.0
Reporter: Vamsee Yarlagadda
Priority: Minor
Attachments: SOLR-7177.patch, SOLR-7177v2.patch
I notice when there is an http connection failure, we simply log the error
but not the connection information. It would be good to log this info to make
debugging easier.
e.g:
1.
{code}
2015-02-27 08:56:51,503 ERROR org.apache.solr.update.StreamingSolrServers:
error
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:235)
{code}
2.
{code}
2015-02-27 10:26:12,363 ERROR org.apache.solr.update.StreamingSolrServers:
error
org.apache.http.NoHttpResponseException: The target server failed to respond
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
[jira] [Commented] (LUCENE-6304) Add MatchNoDocsQuery that matches no documents
[ https://issues.apache.org/jira/browse/LUCENE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345352#comment-14345352 ] David Smiley commented on LUCENE-6304: -- +1 Adrien. Add MatchNoDocsQuery that matches no documents -- Key: LUCENE-6304 URL: https://issues.apache.org/jira/browse/LUCENE-6304 Project: Lucene - Core Issue Type: Improvement Components: core/search Affects Versions: 5.0 Reporter: Lee Hinman Priority: Minor Attachments: LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch As a followup to LUCENE-6298, it would be nice to have an explicit MatchNoDocsQuery to indicate that no documents should be matched. This would hopefully be a better indicator than a BooleanQuery with no clauses or (even worse) null. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2709 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2709/ 5 tests failed. FAILED: org.apache.solr.cloud.HttpPartitionTest.test Error Message: org.apache.http.NoHttpResponseException: The target server failed to respond Stack Trace: org.apache.solr.client.solrj.SolrServerException: org.apache.http.NoHttpResponseException: The target server failed to respond at __randomizedtesting.SeedInfo.seed([412A5FBE3443EEFA:C97E60649ABF8302]:0) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:899) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:752) at org.apache.solr.cloud.HttpPartitionTest.doSendDoc(HttpPartitionTest.java:484) at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:501) at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:193) at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:950) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:925) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at
[jira] [Commented] (SOLR-7185) null pointer exception at AbstractSolrSecondaryIndex.java
[ https://issues.apache.org/jira/browse/SOLR-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345369#comment-14345369 ] MASSIMO CELLI commented on SOLR-7185: - apologies I created this in error on the Solr open source jira null pointer exception at AbstractSolrSecondaryIndex.java -- Key: SOLR-7185 URL: https://issues.apache.org/jira/browse/SOLR-7185 Project: Solr Issue Type: Bug Components: search Affects Versions: 4.6 Reporter: MASSIMO CELLI Priority: Minor a few hundreds of the following NPE were reported in the system.log: INFO [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,435 AbstractSolrSecondaryIndex.java (line 1491) Found 4 rows with expired columns. WARN [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,436 AbstractSolrSecondaryIndex.java (line 1505) java.lang.NullPointerException at com.datastax.bdp.search.solr.Cql3SolrSecondaryIndex.buildIndexedColumnFamily(Cql3SolrSecondaryIndex.java:98) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.doApplyUpdates(AbstractSolrSecondaryIndex.java:932) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.access$1300(AbstractSolrSecondaryIndex.java:94) at com .datastax.bdp.search.solr.AbstractSolrSecondaryIndex$TTLIndexRebuildTask.run(AbstractSolrSecondaryIndex.java:1501) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) WARN [SolrSecondaryIndex ax.account_freq_accessed TTL scheduler.] 2015-03-02 17:38:19,436 AbstractSolrSecondaryIndex.java (line 1505) java.lang.NullPointerException at com.datastax.bdp.search.solr.Cql3SolrSecondaryIndex.buildIndexedColumnFamily(Cql3SolrSecondaryIndex.java:98) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.doApplyUpdates(AbstractSolrSecondaryIndex.java:932) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex.access$1300(AbstractSolrSecondaryIndex.java:94) at com.datastax.bdp.search.solr.AbstractSolrSecondaryIndex$TTLIndexRebuildTask.run(AbstractSolrSecondaryIndex.java:1501) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7174) DIH can't use TikaEntityProcessor as inner entity because it's not capable of re-entry.
[
https://issues.apache.org/jira/browse/SOLR-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345516#comment-14345516
]
Noble Paul commented on SOLR-7174:
--
The patch looks fine to me , I shall commit this soon
DIH can't use TikaEntityProcessor as inner entity because it's not capable of
re-entry.
---
Key: SOLR-7174
URL: https://issues.apache.org/jira/browse/SOLR-7174
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 5.0
Environment: Windows 7. Ubuntu 14.04.
Reporter: Gary Taylor
Labels: dataimportHandler, tika,text-extraction
Attachments: SOLR-7174.patch
Downloaded Solr 5.0.0, on a Windows 7 PC. I ran solr start and then solr
create -c hn2 to create a new core.
I want to index a load of epub files that I've got in a directory. So I
created a data-import.xml (in solr\hn2\conf):
dataConfig
dataSource type=BinFileDataSource name=bin /
document
entity name=files dataSource=null rootEntity=false
processor=FileListEntityProcessor
baseDir=c:/Users/gt/Documents/epub fileName=.*epub
onError=skip
recursive=true
field column=fileAbsolutePath name=id /
field column=fileSize name=size /
field column=fileLastModified name=lastModified /
entity name=documentImport processor=TikaEntityProcessor
url=${files.fileAbsolutePath} format=text
dataSource=bin onError=skip
field column=file name=fileName/
field column=Author name=author meta=true/
field column=title name=title meta=true/
field column=text name=content/
/entity
/entity
/document
/dataConfig
In my solrconfig.xml, I added a requestHandler entry to reference my
data-import.xml:
requestHandler name=/dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=configdata-import.xml/str
/lst
/requestHandler
I renamed managed-schema to schema.xml, and ensured the following doc fields
were setup:
field name=id type=string indexed=true stored=true
required=true multiValued=false /
field name=fileName type=string indexed=true stored=true /
field name=author type=string indexed=true stored=true /
field name=title type=string indexed=true stored=true /
field name=size type=long indexed=true stored=true /
field name=lastModified type=date indexed=true stored=true /
field name=content type=text_en indexed=false stored=true
multiValued=false/
field name=text type=text_en indexed=true stored=false
multiValued=true/
copyField source=content dest=text/
I copied all the jars from dist and contrib\* into server\solr\lib.
Stopping and restarting solr then creates a new managed-schema file and
renames schema.xml to schema.xml.back
All good so far.
Now I go to the web admin for dataimport
(http://localhost:8983/solr/#/hn2/dataimport//dataimport) and try and execute
a full import.
But, the results show Requests: 0, Fetched: 58, Skipped: 0, Processed:1 -
ie. it only adds one document (the very first one) even though it's iterated
over 58!
No errors are reported in the logs.
I can repeat this on Ubuntu 14.04 using the same steps, so it's not Windows
specific.
-
If I change the data-import.xml to use FileDataSource and
PlainTextEntityProcessor and parse txt files, eg:
dataConfig
dataSource type=FileDataSource name=bin /
document
entity name=files dataSource=null rootEntity=false
processor=FileListEntityProcessor
baseDir=c:/Users/gt/Documents/epub fileName=.*txt
field column=fileAbsolutePath name=id /
field column=fileSize name=size /
field column=fileLastModified name=lastModified /
entity name=documentImport
processor=PlainTextEntityProcessor
url=${files.fileAbsolutePath} format=text
dataSource=bin
field column=plainText name=content/
/entity
/entity
/document
/dataConfig
This works. So it's a combo of BinFileDataSource and TikaEntityProcessor
that is failing.
On Windows, I ran Process Monitor, and spotted that only the very first epub
file is actually being read (repeatedly).
With verbose and debug on when running the DIH, I get the following response:
verbose-output: [
entity:files,
[
null,
--- row
[jira] [Assigned] (SOLR-7174) DIH can't use TikaEntityProcessor as inner entity because it's not capable of re-entry.
[
https://issues.apache.org/jira/browse/SOLR-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul reassigned SOLR-7174:
Assignee: Noble Paul
DIH can't use TikaEntityProcessor as inner entity because it's not capable of
re-entry.
---
Key: SOLR-7174
URL: https://issues.apache.org/jira/browse/SOLR-7174
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 5.0
Environment: Windows 7. Ubuntu 14.04.
Reporter: Gary Taylor
Assignee: Noble Paul
Labels: dataimportHandler, tika,text-extraction
Attachments: SOLR-7174.patch
Downloaded Solr 5.0.0, on a Windows 7 PC. I ran solr start and then solr
create -c hn2 to create a new core.
I want to index a load of epub files that I've got in a directory. So I
created a data-import.xml (in solr\hn2\conf):
dataConfig
dataSource type=BinFileDataSource name=bin /
document
entity name=files dataSource=null rootEntity=false
processor=FileListEntityProcessor
baseDir=c:/Users/gt/Documents/epub fileName=.*epub
onError=skip
recursive=true
field column=fileAbsolutePath name=id /
field column=fileSize name=size /
field column=fileLastModified name=lastModified /
entity name=documentImport processor=TikaEntityProcessor
url=${files.fileAbsolutePath} format=text
dataSource=bin onError=skip
field column=file name=fileName/
field column=Author name=author meta=true/
field column=title name=title meta=true/
field column=text name=content/
/entity
/entity
/document
/dataConfig
In my solrconfig.xml, I added a requestHandler entry to reference my
data-import.xml:
requestHandler name=/dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=configdata-import.xml/str
/lst
/requestHandler
I renamed managed-schema to schema.xml, and ensured the following doc fields
were setup:
field name=id type=string indexed=true stored=true
required=true multiValued=false /
field name=fileName type=string indexed=true stored=true /
field name=author type=string indexed=true stored=true /
field name=title type=string indexed=true stored=true /
field name=size type=long indexed=true stored=true /
field name=lastModified type=date indexed=true stored=true /
field name=content type=text_en indexed=false stored=true
multiValued=false/
field name=text type=text_en indexed=true stored=false
multiValued=true/
copyField source=content dest=text/
I copied all the jars from dist and contrib\* into server\solr\lib.
Stopping and restarting solr then creates a new managed-schema file and
renames schema.xml to schema.xml.back
All good so far.
Now I go to the web admin for dataimport
(http://localhost:8983/solr/#/hn2/dataimport//dataimport) and try and execute
a full import.
But, the results show Requests: 0, Fetched: 58, Skipped: 0, Processed:1 -
ie. it only adds one document (the very first one) even though it's iterated
over 58!
No errors are reported in the logs.
I can repeat this on Ubuntu 14.04 using the same steps, so it's not Windows
specific.
-
If I change the data-import.xml to use FileDataSource and
PlainTextEntityProcessor and parse txt files, eg:
dataConfig
dataSource type=FileDataSource name=bin /
document
entity name=files dataSource=null rootEntity=false
processor=FileListEntityProcessor
baseDir=c:/Users/gt/Documents/epub fileName=.*txt
field column=fileAbsolutePath name=id /
field column=fileSize name=size /
field column=fileLastModified name=lastModified /
entity name=documentImport
processor=PlainTextEntityProcessor
url=${files.fileAbsolutePath} format=text
dataSource=bin
field column=plainText name=content/
/entity
/entity
/document
/dataConfig
This works. So it's a combo of BinFileDataSource and TikaEntityProcessor
that is failing.
On Windows, I ran Process Monitor, and spotted that only the very first epub
file is actually being read (repeatedly).
With verbose and debug on when running the DIH, I get the following response:
verbose-output: [
entity:files,
[
null,
--- row #1-,
fileSize,
2609004,
[jira] [Resolved] (LUCENE-6329) Calling score() should be ok even if needsScores is false
[ https://issues.apache.org/jira/browse/LUCENE-6329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand resolved LUCENE-6329. -- Resolution: Fixed Calling score() should be ok even if needsScores is false - Key: LUCENE-6329 URL: https://issues.apache.org/jira/browse/LUCENE-6329 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: Trunk, 5.1 Attachments: LUCENE-6329.patch Currently if you use approximations on a disjunction scorer and call score() while needsScores was set to false, then you might run into exceptions. The reason is that when scores are not needed then the scorer doesn't maintain the list of scorers positioned on the current doc ID and you can eg. end up with exhausted scorers in this list. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6331) Spans for SpanOrQuery can return 0 cost
Paul Elschot created LUCENE-6331: Summary: Spans for SpanOrQuery can return 0 cost Key: LUCENE-6331 URL: https://issues.apache.org/jira/browse/LUCENE-6331 Project: Lucene - Core Issue Type: Bug Components: core/search Reporter: Paul Elschot On the Spans for SpanOrQuery, cost() can be called before the span queue is initialized, and the total cost is only computed when the queue is initialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6331) Spans for SpanOrQuery can return 0 cost
[ https://issues.apache.org/jira/browse/LUCENE-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Elschot updated LUCENE-6331: - Lucene Fields: New,Patch Available (was: New) Spans for SpanOrQuery can return 0 cost --- Key: LUCENE-6331 URL: https://issues.apache.org/jira/browse/LUCENE-6331 Project: Lucene - Core Issue Type: Bug Components: core/search Reporter: Paul Elschot Attachments: LUCENE-6331.patch On the Spans for SpanOrQuery, cost() can be called before the span queue is initialized, and the total cost is only computed when the queue is initialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7180) MiniSolrCloudCluster should startup and shutdown its jetties in parallel
[ https://issues.apache.org/jira/browse/SOLR-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345426#comment-14345426 ] Tomás Fernández Löbbe commented on SOLR-7180: - Should we awaitTermination in shutdown too? Also, is there a reason why you don't call stopJettySolrRunner? MiniSolrCloudCluster should startup and shutdown its jetties in parallel Key: SOLR-7180 URL: https://issues.apache.org/jira/browse/SOLR-7180 Project: Solr Issue Type: Improvement Reporter: Alan Woodward Assignee: Alan Woodward Priority: Minor Attachments: SOLR-7180.patch Followup to SOLR-7179. Now JettySolrRunner doesn't use sysprops to pass configuration, we can start up multiple runners in parallel. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Artifacts-5.0 - Build # 46 - Failure
Build: https://builds.apache.org/job/Lucene-Artifacts-5.0/46/ No tests ran. Build Log: [...truncated 12009 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Artifacts-5.0/lucene/build.xml:389: java.net.ConnectException: Operation timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.init(HttpClient.java:211) at sun.net.www.http.HttpClient.New(HttpClient.java:308) at sun.net.www.http.HttpClient.New(HttpClient.java:326) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850) at org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660) at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579) at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569) Total time: 7 minutes 27 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Sending artifact delta relative to Lucene-Artifacts-5.0 #45 Archived 13 artifacts Archive block size is 32768 Received 1142 blocks and 137404243 bytes Compression is 21.4% Took 14 min Publishing Javadoc Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-6331) Spans for SpanOrQuery can return 0 cost
[ https://issues.apache.org/jira/browse/LUCENE-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345631#comment-14345631 ] Paul Elschot edited comment on LUCENE-6331 at 3/3/15 7:58 PM: -- The bug can be checked by adding an assert queue != null to the cost() function at SpanOrQuery.getSpans(). This test case then fails: ant test -Dtestcase=TestSpanMultiTermQueryWrapper -Dtests.method=testNoSuchMultiTermsInOr -Dtests.seed=D2F850A1F724092F was (Author: paul.elsc...@xs4all.nl): The bug can be checked by addingan assert queue != null to the cost() function at SpanOrQuery.getSpans(). This test case then fails: ant test -Dtestcase=TestSpanMultiTermQueryWrapper -Dtests.method=testNoSuchMultiTermsInOr -Dtests.seed=D2F850A1F724092F Spans for SpanOrQuery can return 0 cost --- Key: LUCENE-6331 URL: https://issues.apache.org/jira/browse/LUCENE-6331 Project: Lucene - Core Issue Type: Bug Components: core/search Reporter: Paul Elschot Attachments: LUCENE-6331.patch On the Spans for SpanOrQuery, cost() can be called before the span queue is initialized, and the total cost is only computed when the queue is initialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6331) Spans for SpanOrQuery can return 0 cost
[ https://issues.apache.org/jira/browse/LUCENE-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Elschot updated LUCENE-6331: - Attachment: LUCENE-6331.patch Spans for SpanOrQuery can return 0 cost --- Key: LUCENE-6331 URL: https://issues.apache.org/jira/browse/LUCENE-6331 Project: Lucene - Core Issue Type: Bug Components: core/search Reporter: Paul Elschot Attachments: LUCENE-6331.patch On the Spans for SpanOrQuery, cost() can be called before the span queue is initialized, and the total cost is only computed when the queue is initialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6331) Spans for SpanOrQuery can return 0 cost
[ https://issues.apache.org/jira/browse/LUCENE-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345636#comment-14345636 ] Paul Elschot commented on LUCENE-6331: -- Actually, for that test case, the currently returned 0 may be correct, so I'm not sure that therereally is a bug. Spans for SpanOrQuery can return 0 cost --- Key: LUCENE-6331 URL: https://issues.apache.org/jira/browse/LUCENE-6331 Project: Lucene - Core Issue Type: Bug Components: core/search Reporter: Paul Elschot Attachments: LUCENE-6331.patch On the Spans for SpanOrQuery, cost() can be called before the span queue is initialized, and the total cost is only computed when the queue is initialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6329) Calling score() should be ok even if needsScores is false
[ https://issues.apache.org/jira/browse/LUCENE-6329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345414#comment-14345414 ] ASF subversion and git services commented on LUCENE-6329: - Commit 1663756 from [~jpountz] in branch 'dev/trunk' [ https://svn.apache.org/r1663756 ] LUCENE-6329: Calling score() should be ok even if needsScores is false. Calling score() should be ok even if needsScores is false - Key: LUCENE-6329 URL: https://issues.apache.org/jira/browse/LUCENE-6329 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: Trunk, 5.1 Attachments: LUCENE-6329.patch Currently if you use approximations on a disjunction scorer and call score() while needsScores was set to false, then you might run into exceptions. The reason is that when scores are not needed then the scorer doesn't maintain the list of scorers positioned on the current doc ID and you can eg. end up with exhausted scorers in this list. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 2027 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2027/ Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC 1 tests failed. FAILED: org.apache.solr.schema.TestCloudManagedSchemaConcurrent.test Error Message: The target server failed to respond Stack Trace: org.apache.http.NoHttpResponseException: The target server failed to respond at __randomizedtesting.SeedInfo.seed([C2FC830A1FEAECC9:4AA8BCD0B1168131]:0) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251) at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106) at org.apache.solr.util.RestTestHarness.getResponse(RestTestHarness.java:208) at org.apache.solr.util.RestTestHarness.query(RestTestHarness.java:100) at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.schemaLockTest(TestCloudManagedSchemaConcurrent.java:677) at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.test(TestCloudManagedSchemaConcurrent.java:253) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:950) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:925) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Updated] (LUCENE-6304) Add MatchNoDocsQuery that matches no documents
[ https://issues.apache.org/jira/browse/LUCENE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lee Hinman updated LUCENE-6304: --- Attachment: LUCENE-6304.patch Adrien: I agree about having the hashCode. Here is a new patch that doesn't override equals or hashCode and changes Query to use use the class in the hashCode method as Adrien suggested. Add MatchNoDocsQuery that matches no documents -- Key: LUCENE-6304 URL: https://issues.apache.org/jira/browse/LUCENE-6304 Project: Lucene - Core Issue Type: Improvement Components: core/search Affects Versions: 5.0 Reporter: Lee Hinman Priority: Minor Attachments: LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch As a followup to LUCENE-6298, it would be nice to have an explicit MatchNoDocsQuery to indicate that no documents should be matched. This would hopefully be a better indicator than a BooleanQuery with no clauses or (even worse) null. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-6304) Add MatchNoDocsQuery that matches no documents
[ https://issues.apache.org/jira/browse/LUCENE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345602#comment-14345602 ] Lee Hinman edited comment on LUCENE-6304 at 3/3/15 7:42 PM: Adrien: I agree about having the hashCode. Here is a new patch that doesn't override equals or hashCode and changes Query to use the class in the hashCode method as Adrien suggested. was (Author: dakrone): Adrien: I agree about having the hashCode. Here is a new patch that doesn't override equals or hashCode and changes Query to use use the class in the hashCode method as Adrien suggested. Add MatchNoDocsQuery that matches no documents -- Key: LUCENE-6304 URL: https://issues.apache.org/jira/browse/LUCENE-6304 Project: Lucene - Core Issue Type: Improvement Components: core/search Affects Versions: 5.0 Reporter: Lee Hinman Priority: Minor Attachments: LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch As a followup to LUCENE-6298, it would be nice to have an explicit MatchNoDocsQuery to indicate that no documents should be matched. This would hopefully be a better indicator than a BooleanQuery with no clauses or (even worse) null. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6331) Spans for SpanOrQuery can return 0 cost
[ https://issues.apache.org/jira/browse/LUCENE-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Elschot updated LUCENE-6331: - Priority: Minor (was: Major) Spans for SpanOrQuery can return 0 cost --- Key: LUCENE-6331 URL: https://issues.apache.org/jira/browse/LUCENE-6331 Project: Lucene - Core Issue Type: Bug Components: core/search Reporter: Paul Elschot Priority: Minor Attachments: LUCENE-6331.patch On the Spans for SpanOrQuery, cost() can be called before the span queue is initialized, and the total cost is only computed when the queue is initialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6329) Calling score() should be ok even if needsScores is false
[ https://issues.apache.org/jira/browse/LUCENE-6329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345420#comment-14345420 ] ASF subversion and git services commented on LUCENE-6329: - Commit 1663757 from [~jpountz] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1663757 ] LUCENE-6329: Calling score() should be ok even if needsScores is false. Calling score() should be ok even if needsScores is false - Key: LUCENE-6329 URL: https://issues.apache.org/jira/browse/LUCENE-6329 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: Trunk, 5.1 Attachments: LUCENE-6329.patch Currently if you use approximations on a disjunction scorer and call score() while needsScores was set to false, then you might run into exceptions. The reason is that when scores are not needed then the scorer doesn't maintain the list of scorers positioned on the current doc ID and you can eg. end up with exhausted scorers in this list. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6331) Spans for SpanOrQuery can return 0 cost
[ https://issues.apache.org/jira/browse/LUCENE-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345631#comment-14345631 ] Paul Elschot commented on LUCENE-6331: -- The bug can be checked by addingan assert queue != null to the cost() function at SpanOrQuery.getSpans(). This test case then fails: ant test -Dtestcase=TestSpanMultiTermQueryWrapper -Dtests.method=testNoSuchMultiTermsInOr -Dtests.seed=D2F850A1F724092F Spans for SpanOrQuery can return 0 cost --- Key: LUCENE-6331 URL: https://issues.apache.org/jira/browse/LUCENE-6331 Project: Lucene - Core Issue Type: Bug Components: core/search Reporter: Paul Elschot On the Spans for SpanOrQuery, cost() can be called before the span queue is initialized, and the total cost is only computed when the queue is initialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6304) Add MatchNoDocsQuery that matches no documents
[ https://issues.apache.org/jira/browse/LUCENE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345707#comment-14345707 ] Robert Muir commented on LUCENE-6304: - Thanks Lee, I like this. When i see code overriding hashcode/equals and not calling super.hashcode/super.equals, its a bad sign. We should commit this one, and remove duplicated logic and hardcoded constants in e.g. TermQuery and all other places in a followup. Add MatchNoDocsQuery that matches no documents -- Key: LUCENE-6304 URL: https://issues.apache.org/jira/browse/LUCENE-6304 Project: Lucene - Core Issue Type: Improvement Components: core/search Affects Versions: 5.0 Reporter: Lee Hinman Priority: Minor Attachments: LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch As a followup to LUCENE-6298, it would be nice to have an explicit MatchNoDocsQuery to indicate that no documents should be matched. This would hopefully be a better indicator than a BooleanQuery with no clauses or (even worse) null. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4408) Server hanging on startup
[
https://issues.apache.org/jira/browse/SOLR-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345716#comment-14345716
]
Michael Wilken commented on SOLR-4408:
--
I am seeing this issue with Solr 4.10.2. Having collate set to 'true' in the
/select requestHandler causes Solr to hang when starting up. Removing the
collate option returns everything to normal. My entire spellchecking config:
{code}
str name=spellcheckon/str
str name=spellcheck.extendedResultstrue/str
str name=spellcheck.count5/str
str name=spellcheck.alternativeTermCount2/str
str name=spellcheck.maxResultsForSuggest100/str
str name=spellcheck.collatetrue/str
str name=spellcheck.collateExtendedResultstrue/str
str name=spellcheck.maxCollationTries5/str
str name=spellcheck.maxCollations3/str
str name=spellcheck.onlyMorePopulartrue/str
{code}
Server hanging on startup
-
Key: SOLR-4408
URL: https://issues.apache.org/jira/browse/SOLR-4408
Project: Solr
Issue Type: Bug
Affects Versions: 4.1
Environment: OpenJDK 64-Bit Server VM (23.2-b09 mixed mode)
Tomcat 7.0
Eclipse Juno + WTP
Reporter: Francois-Xavier Bonnet
Attachments: patch-4408.txt
While starting, the server hangs indefinitely. Everything works fine when I
first start the server with no index created yet but if I fill the index then
stop and start the server, it hangs. Could it be a lock that is never
released?
Here is what I get in a full thread dump:
2013-02-06 16:28:52
Full thread dump OpenJDK 64-Bit Server VM (23.2-b09 mixed mode):
searcherExecutor-4-thread-1 prio=10 tid=0x7fbdfc16a800 nid=0x42c6 in
Object.wait() [0x7fbe0ab1]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xc34c1c48 (a java.lang.Object)
at java.lang.Object.wait(Object.java:503)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1492)
- locked 0xc34c1c48 (a java.lang.Object)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1312)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1247)
at
org.apache.solr.request.SolrQueryRequestBase.getSearcher(SolrQueryRequestBase.java:94)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:213)
at
org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:112)
at
org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:203)
at
org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:180)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1816)
at
org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64)
at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1594)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
coreLoadExecutor-3-thread-1 prio=10 tid=0x7fbe04194000 nid=0x42c5 in
Object.wait() [0x7fbe0ac11000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xc34c1c48 (a java.lang.Object)
at java.lang.Object.wait(Object.java:503)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1492)
- locked 0xc34c1c48 (a java.lang.Object)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1312)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1247)
at
org.apache.solr.handler.ReplicationHandler.getIndexVersion(ReplicationHandler.java:495)
at
org.apache.solr.handler.ReplicationHandler.getStatistics(ReplicationHandler.java:518)
at
org.apache.solr.core.JmxMonitoredMap$SolrDynamicMBean.getMBeanInfo(JmxMonitoredMap.java:232)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(DefaultMBeanServerInterceptor.java:333)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:319)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:512)
at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
at
[jira] [Commented] (LUCENE-6304) Add MatchNoDocsQuery that matches no documents
[ https://issues.apache.org/jira/browse/LUCENE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345820#comment-14345820 ] Lee Hinman commented on LUCENE-6304: Robert: +1, I opened LUCENE-6333 for this, I'll work on a patch. Add MatchNoDocsQuery that matches no documents -- Key: LUCENE-6304 URL: https://issues.apache.org/jira/browse/LUCENE-6304 Project: Lucene - Core Issue Type: Improvement Components: core/search Affects Versions: 5.0 Reporter: Lee Hinman Priority: Minor Attachments: LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch, LUCENE-6304.patch As a followup to LUCENE-6298, it would be nice to have an explicit MatchNoDocsQuery to indicate that no documents should be matched. This would hopefully be a better indicator than a BooleanQuery with no clauses or (even worse) null. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7186) Errors reported in GUI should clear - either manually or by taking action to fix them
Garth Grimm created SOLR-7186: - Summary: Errors reported in GUI should clear - either manually or by taking action to fix them Key: SOLR-7186 URL: https://issues.apache.org/jira/browse/SOLR-7186 Project: Solr Issue Type: Improvement Components: web gui Reporter: Garth Grimm Priority: Minor The red banner text for notification of some ERRORS on the Solr admin GUI is nice, but there needs to be some way to make it disappear after the admin feels the issue has been resolved. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6332) JoinUtil.createSemiJoinQuery()
[
https://issues.apache.org/jira/browse/LUCENE-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345689#comment-14345689
]
Mikhail Khludnev commented on LUCENE-6332:
--
might be made available for Solr as a query parser like SOLR-6234
JoinUtil.createSemiJoinQuery()
--
Key: LUCENE-6332
URL: https://issues.apache.org/jira/browse/LUCENE-6332
Project: Lucene - Core
Issue Type: New Feature
Components: modules/join
Affects Versions: 5.0
Reporter: Mikhail Khludnev
I want to contribute the subj which should do something like ..{{WHERE EXISTS
(SELECT 1 FROM fromSearcher.search(fromQuery) WHERE fromField=toField)}}. It
turns out, that it can be returned by the current method
{{createJoinQuery(...ScoreMode.None)}}
* at first, it should run {{fromQuery}} first, collect {{fromField}} into
{{BytesRefHash}} by {{TermsCollector}}, like it's done now
* then it should return query with _TwoPhase_ Scorer
* which obtains {{toField}} on {{matches()}} and check term for existence in
{{BytesRefHash}}
Do you think it's ever useful? if you do, I can bake a patch.
Anyway, suggest the better API eg separate method, or enum and actual name!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4787) Join Contrib
[
https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345727#comment-14345727
]
Bill Bell commented on SOLR-4787:
-
To be consistent can we add FQ?
Based on post by Yonik:
The join qparser has no fq parameter, so that is ignored.
-Yonik
http://heliosearch.org - native code faceting, facet functions,
sub-facets, off-heap data
Join Contrib
Key: SOLR-4787
URL: https://issues.apache.org/jira/browse/SOLR-4787
Project: Solr
Issue Type: New Feature
Components: search
Affects Versions: 4.2.1
Reporter: Joel Bernstein
Priority: Minor
Fix For: Trunk
Attachments: SOLR-4787-deadlock-fix.patch,
SOLR-4787-pjoin-long-keys.patch, SOLR-4787-with-testcase-fix.patch,
SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
SOLR-4787.patch, SOLR-4787.patch,
SOLR-4797-hjoin-multivaluekeys-nestedJoins.patch,
SOLR-4797-hjoin-multivaluekeys-trunk.patch
This contrib provides a place where different join implementations can be
contributed to Solr. This contrib currently includes 3 join implementations.
The initial patch was generated from the Solr 4.3 tag. Because of changes in
the FieldCache API this patch will only build with Solr 4.2 or above.
*HashSetJoinQParserPlugin aka hjoin*
The hjoin provides a join implementation that filters results in one core
based on the results of a search in another core. This is similar in
functionality to the JoinQParserPlugin but the implementation differs in a
couple of important ways.
The first way is that the hjoin is designed to work with int and long join
keys only. So, in order to use hjoin, int or long join keys must be included
in both the to and from core.
The second difference is that the hjoin builds memory structures that are
used to quickly connect the join keys. So, the hjoin will need more memory
then the JoinQParserPlugin to perform the join.
The main advantage of the hjoin is that it can scale to join millions of keys
between cores and provide sub-second response time. The hjoin should work
well with up to two million results from the fromIndex and tens of millions
of results from the main query.
The hjoin supports the following features:
1) Both lucene query and PostFilter implementations. A *cost* 99 will
turn on the PostFilter. The PostFilter will typically outperform the Lucene
query when the main query results have been narrowed down.
2) With the lucene query implementation there is an option to build the
filter with threads. This can greatly improve the performance of the query if
the main query index is very large. The threads parameter turns on
threading. For example *threads=6* will use 6 threads to build the filter.
This will setup a fixed threadpool with six threads to handle all hjoin
requests. Once the threadpool is created the hjoin will always use it to
build the filter. Threading does not come into play with the PostFilter.
3) The *size* local parameter can be used to set the initial size of the
hashset used to perform the join. If this is set above the number of results
from the fromIndex then the you can avoid hashset resizing which improves
performance.
4) Nested filter queries. The local parameter fq can be used to nest a
filter query within the join. The nested fq will filter the results of the
join query. This can point to another join to support nested joins.
5) Full caching support for the lucene query implementation. The filterCache
and queryResultCache should work properly even with deep nesting of joins.
Only the queryResultCache comes into play with the PostFilter implementation
because PostFilters are not cacheable in the filterCache.
The syntax of the hjoin is similar to the JoinQParserPlugin except that the
plugin is referenced by the string hjoin rather then join.
fq=\{!hjoin fromIndex=collection2 from=id_i to=id_i threads=6
fq=$qq\}user:customer1qq=group:5
The example filter query above will search the fromIndex (collection2) for
user:customer1 applying the local fq parameter to filter the results. The
lucene filter query will be built using 6 threads. This query will generate a
list of values from the from field that will be used to filter the main
query. Only records from the main query, where the to field is present in
the from list will be included in the results.
The solrconfig.xml in the main query core must contain the reference to the
hjoin.
queryParser name=hjoin
class=org.apache.solr.joins.HashSetJoinQParserPlugin/
And the join contrib lib jars must be registed in the solrconfig.xml.
lib dir=../../../contrib/joins/lib
[jira] [Created] (LUCENE-6332) JoinUtil.createSemiJoinQuery()
Mikhail Khludnev created LUCENE-6332:
Summary: JoinUtil.createSemiJoinQuery()
Key: LUCENE-6332
URL: https://issues.apache.org/jira/browse/LUCENE-6332
Project: Lucene - Core
Issue Type: New Feature
Components: modules/join
Affects Versions: 5.0
Reporter: Mikhail Khludnev
I want to contribute the subj which should do something like ..{{WHERE EXISTS
(SELECT 1 FROM fromSearcher.search(fromQuery) WHERE fromField=toField)}}. It
turns out, that it can be returned by the current method
{{createJoinQuery(...ScoreMode.None)}}
* at first, it should run {{fromQuery}} first, collect {{fromField}} into
{{BytesRefHash}} by {{TermsCollector}}, like it's done now
* then it should return query with _TwoPhase_ Scorer
* which obtains {{toField}} on {{matches()}} and check term for existence in
{{BytesRefHash}}
Do you think it's ever useful? if you do, I can bake a patch.
Anyway, suggest the better API eg separate method, or enum and actual name!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4408) Server hanging on startup
[
https://issues.apache.org/jira/browse/SOLR-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345716#comment-14345716
]
Michael Wilken edited comment on SOLR-4408 at 3/3/15 9:09 PM:
--
I am seeing this issue with Solr 4.10.2. Having collate set to 'true' in the
/select requestHandler causes Solr to hang when starting up. Removing the
collate option returns everything to normal. My entire spellchecking config:
{code}
str name=spellcheckon/str
str name=spellcheck.extendedResultstrue/str
str name=spellcheck.count5/str
str name=spellcheck.alternativeTermCount2/str
str name=spellcheck.maxResultsForSuggest100/str
str name=spellcheck.collatetrue/str
str name=spellcheck.collateExtendedResultstrue/str
str name=spellcheck.maxCollationTries5/str
str name=spellcheck.maxCollations3/str
str name=spellcheck.onlyMorePopulartrue/str
{code}
When running in a single machine, we see normal behavior. We only see the
'hangs at startup' behavior when running in a 'SolrCloud' setup with 1 shard
replicated across 3 servers.
was (Author: wilkenm):
I am seeing this issue with Solr 4.10.2. Having collate set to 'true' in the
/select requestHandler causes Solr to hang when starting up. Removing the
collate option returns everything to normal. My entire spellchecking config:
{code}
str name=spellcheckon/str
str name=spellcheck.extendedResultstrue/str
str name=spellcheck.count5/str
str name=spellcheck.alternativeTermCount2/str
str name=spellcheck.maxResultsForSuggest100/str
str name=spellcheck.collatetrue/str
str name=spellcheck.collateExtendedResultstrue/str
str name=spellcheck.maxCollationTries5/str
str name=spellcheck.maxCollations3/str
str name=spellcheck.onlyMorePopulartrue/str
{code}
Server hanging on startup
-
Key: SOLR-4408
URL: https://issues.apache.org/jira/browse/SOLR-4408
Project: Solr
Issue Type: Bug
Affects Versions: 4.1
Environment: OpenJDK 64-Bit Server VM (23.2-b09 mixed mode)
Tomcat 7.0
Eclipse Juno + WTP
Reporter: Francois-Xavier Bonnet
Attachments: patch-4408.txt
While starting, the server hangs indefinitely. Everything works fine when I
first start the server with no index created yet but if I fill the index then
stop and start the server, it hangs. Could it be a lock that is never
released?
Here is what I get in a full thread dump:
2013-02-06 16:28:52
Full thread dump OpenJDK 64-Bit Server VM (23.2-b09 mixed mode):
searcherExecutor-4-thread-1 prio=10 tid=0x7fbdfc16a800 nid=0x42c6 in
Object.wait() [0x7fbe0ab1]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xc34c1c48 (a java.lang.Object)
at java.lang.Object.wait(Object.java:503)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1492)
- locked 0xc34c1c48 (a java.lang.Object)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1312)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1247)
at
org.apache.solr.request.SolrQueryRequestBase.getSearcher(SolrQueryRequestBase.java:94)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:213)
at
org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:112)
at
org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:203)
at
org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:180)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1816)
at
org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64)
at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1594)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
coreLoadExecutor-3-thread-1 prio=10 tid=0x7fbe04194000 nid=0x42c5 in
Object.wait() [0x7fbe0ac11000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xc34c1c48 (a java.lang.Object)
at java.lang.Object.wait(Object.java:503)
at
Re: Welcome Ramkumar Aiyengar as Lucene/Solr committer
Welcome! - Mark On Tue, Mar 3, 2015 at 12:07 AM Shai Erera ser...@gmail.com wrote: Welcome On Mar 3, 2015 8:57 AM, Noble Paul noble.p...@gmail.com wrote: Awesome , It was long overdue welcome ramkumar On Tue, Mar 3, 2015 at 12:39 AM, Tomás Fernández Löbbe tomasflo...@gmail.com wrote: Welcome Ramkumar! On Mon, Mar 2, 2015 at 8:01 AM, Ryan Ernst r...@iernst.net wrote: Welcome! On Mar 1, 2015 8:40 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: I'm pleased to announce that Ramkumar Aiyengar has accepted the PMC's invitation to become a committer. Ramkumar, it's tradition that you introduce yourself with a brief bio. Your handle andyetitmoves has already added to the “lucene LDAP group, so you now have commit privileges. Please test this by adding yourself to the committers section of the Who We Are page on the website: http://lucene.apache.org/whoweare.html (use the ASF CMS bookmarklet at the bottom of the page here: https://cms.apache.org/#bookmark - more info here http://www.apache.org/dev/cms.html). The ASF dev page also has lots of useful links: http://www.apache.org/dev/. Congratulations and welcome! -- Regards, Shalin Shekhar Mangar. -- - Noble Paul
[jira] [Created] (LUCENE-6333) Clean up overridden .equals and .hashCode methods in Query subclasses
Lee Hinman created LUCENE-6333:
--
Summary: Clean up overridden .equals and .hashCode methods in
Query subclasses
Key: LUCENE-6333
URL: https://issues.apache.org/jira/browse/LUCENE-6333
Project: Lucene - Core
Issue Type: Improvement
Components: core/search
Affects Versions: 5.0
Reporter: Lee Hinman
Priority: Minor
As a followup to LUCENE-6304, all classes that subclass Query and override the
{{equals}} and {{hashCode}} methods should call super.equals/hashCode and, when
possible, not override the methods at all.
For example, TermQuery.hashCode overrides the Query.hashCode, but will be
exactly the same code once LUCENE-6304 is merged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-7082) Streaming Aggregation for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-7082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344350#comment-14344350 ] Joel Bernstein edited comment on SOLR-7082 at 3/3/15 8:48 PM: -- Patch with all tests passing. was (Author: joel.bernstein): Patch with all test passing. Streaming Aggregation for SolrCloud --- Key: SOLR-7082 URL: https://issues.apache.org/jira/browse/SOLR-7082 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Joel Bernstein Fix For: Trunk, 5.1 Attachments: SOLR-7082.patch, SOLR-7082.patch, SOLR-7082.patch This issue provides a general purpose streaming aggregation framework for SolrCloud. An overview of how it works can be found at this link: http://heliosearch.org/streaming-aggregation-for-solrcloud/ This functionality allows SolrCloud users to perform operations that we're typically done using map/reduce or a parallel computing platform. Here is a brief explanation of how the framework works: There is a new Solrj *io* package found in: *org.apache.solr.client.solrj.io* Key classes: *Tuple*: Abstracts a document in a search result as a Map of key/value pairs. *TupleStream*: is the base class for all of the streams. Abstracts search results as a stream of Tuples. *SolrStream*: connects to a single Solr instance. You call the read() method to iterate over the Tuples. *CloudSolrStream*: connects to a SolrCloud collection and merges the results based on the sort param. The merge takes place in CloudSolrStream itself. *Decorator Streams*: wrap other streams to gather *Metrics* on streams and *transform* the streams. Some examples are the MetricStream, RollupStream, GroupByStream, UniqueStream, MergeJoinStream, HashJoinStream, MergeStream, FilterStream. *Going parallel with the ParallelStream and Worker Collections* The io package also contains the *ParallelStream*, which wraps a TupleStream and sends it to N worker nodes. The workers are chosen from a SolrCloud collection. These Worker Collections don't have to hold any data, they can just be used to execute TupleStreams. *The StreamHandler* The Worker nodes have a new RequestHandler called the *StreamHandler*. The ParallelStream serializes a TupleStream, before it is opened, and sends it to the StreamHandler on the Worker Nodes. The StreamHandler on each Worker node deserializes the TupleStream, opens the stream, iterates the tuples and streams them back to the ParallelStream. The ParallelStream performs the final merge of Metrics and can be wrapped by other Streams to handled the final merged TupleStream. *Sorting and Partitioning search results (Shuffling)* Each Worker node is shuffled 1/N of the document results. There is a partitionKeys parameter that can be included with each TupleStream to ensure that Tuples with the same partitionKeys are shuffled to the same Worker. The actual partitioning is done with a filter query using the HashQParserPlugin. The DocSets from the HashQParserPlugin can be cached in the filter cache which provides extremely high performance hash partitioning. Many of the stream transformations rely on the sort order of the TupleStreams (GroupByStream, MergeJoinStream, UniqueStream, FilterStream etc..). To accommodate this the search results can be sorted by specific keys. The /export handler can be used to sort entire result sets efficiently. By specifying the sort order of the results and the partition keys, documents will be sorted and partitioned inside of the search engine. So when the tuples hit the network they are already sorted, partitioned and headed directly to correct worker node. *Extending The Framework* To extend the framework you create new TupleStream Decorators, that gather custom metrics or perform custom stream transformations. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6335) fail tests on threadlocal leaks
[
https://issues.apache.org/jira/browse/LUCENE-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-6335:
Attachment: LUCENE-6335.patch
updated patch: I started fixing some of the analyzers tests.
fail tests on threadlocal leaks
---
Key: LUCENE-6335
URL: https://issues.apache.org/jira/browse/LUCENE-6335
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-6335.patch, LUCENE-6335.patch
I know this requires us to do things like close our Analyzers in tests, but I
think we should do it. just experimenting i found other leaks, e.g. in
LineFileDocs.
{noformat}
[junit4] ERROR 0.00s J0 | TestForceMergeForever (suite)
[junit4] Throwable #1: java.lang.IllegalStateException: ThreadLocal
leaks were found:
[junit4] 1.
thread=SUITE-TestForceMergeForever-seed#[AF7141C55A57350E]-worker
value=WeakReferenceHashMap?,Analyzer$TokenStreamComponents
[junit4] 2.
thread=SUITE-TestForceMergeForever-seed#[AF7141C55A57350E]-worker
value=LineFileDocs$DocState
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7187) SolrCloud does not fully clean collection after delete
[
https://issues.apache.org/jira/browse/SOLR-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346485#comment-14346485
]
Ramkumar Aiyengar commented on SOLR-7187:
-
Quite possibly different, but is the fix from SOLR-7011 applied?
SolrCloud does not fully clean collection after delete
--
Key: SOLR-7187
URL: https://issues.apache.org/jira/browse/SOLR-7187
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.10.3
Reporter: Mike Drob
When I attempt to delete a collection using
{{/admin/collections?action=DELETEname=collection1}} if I go into HDFS I
will still see remnants from the collection. No files, but empty directories
stick around.
{noformat}
[root@solr1 ~]# sudo -u hdfs hdfs dfs -ls -R /solr/collection1
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node1
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node2
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node3
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node4
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node5
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node6
{noformat}
(Edit: I had the wrong log portion here originally)
In the logs, after deleting all the data, I see:
{noformat}
2015-03-03 16:15:14,762 INFO org.apache.solr.servlet.SolrDispatchFilter:
[admin] webapp=null path=/admin/cores
params={deleteInstanceDir=trueaction=UNLOADcore=collection1_shard5_replica1wt=javabinqt=/admin/coresdeleteDataDir=trueversion=2}
status=0 QTime=362
2015-03-03 16:15:14,787 INFO org.apache.solr.common.cloud.ZkStateReader: A
cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
2015-03-03 16:15:14,854 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:14,879 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:14,896 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:14,920 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:15,151 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:15,170 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:15,279 INFO org.apache.solr.common.cloud.ZkStateReader: A
cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
2015-03-03 16:15:15,546 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path:
/overseer/collection-queue-work/qnr-16 state: SyncConnected type
NodeDataChanged
2015-03-03 16:15:15,562 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/collection-queue-work state:
SyncConnected type NodeChildrenChanged
2015-03-03 16:15:15,562 INFO
org.apache.solr.cloud.OverseerCollectionProcessor: Overseer Collection
Processor: Message id:/overseer/collection-queue-work/qn-16 complete,
response:{success={solr1.example.com:8983_solr={responseHeader={status=0,QTime=207}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=243}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=243}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=342}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=346}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=362
{noformat}
This might be related to SOLR-5023, but I'm not sure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ramkumar Aiyengar as Lucene/Solr committer
Congratulations! Otis On Mar 1, 2015, at 23:39, Shalin Shekhar Mangar shalinman...@gmail.com wrote: I'm pleased to announce that Ramkumar Aiyengar has accepted the PMC's invitation to become a committer. Ramkumar, it's tradition that you introduce yourself with a brief bio. Your handle andyetitmoves has already added to the “lucene LDAP group, so you now have commit privileges. Please test this by adding yourself to the committers section of the Who We Are page on the website: http://lucene.apache.org/whoweare.html (use the ASF CMS bookmarklet at the bottom of the page here: https://cms.apache.org/#bookmark - more info here http://www.apache.org/dev/cms.html). The ASF dev page also has lots of useful links: http://www.apache.org/dev/. Congratulations and welcome! -- Regards, Shalin Shekhar Mangar.
[jira] [Created] (LUCENE-6335) fail tests on threadlocal leaks
Robert Muir created LUCENE-6335:
---
Summary: fail tests on threadlocal leaks
Key: LUCENE-6335
URL: https://issues.apache.org/jira/browse/LUCENE-6335
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
I know this requires us to do things like close our Analyzers in tests, but I
think we should do it. just experimenting i found other leaks, e.g. in
LineFileDocs.
{noformat}
[junit4] ERROR 0.00s J0 | TestForceMergeForever (suite)
[junit4] Throwable #1: java.lang.IllegalStateException: ThreadLocal
leaks were found:
[junit4] 1.
thread=SUITE-TestForceMergeForever-seed#[AF7141C55A57350E]-worker
value=WeakReferenceHashMap?,Analyzer$TokenStreamComponents
[junit4] 2.
thread=SUITE-TestForceMergeForever-seed#[AF7141C55A57350E]-worker
value=LineFileDocs$DocState
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7174) DIH should reset TikaEntityProcessor so that it is capable of re-use.
[
https://issues.apache.org/jira/browse/SOLR-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346488#comment-14346488
]
Noble Paul commented on SOLR-7174:
--
It's not a bug
On Mar 4, 2015 10:48 AM, Shalin Shekhar Mangar (JIRA) j...@apache.org
DIH should reset TikaEntityProcessor so that it is capable of re-use.
-
Key: SOLR-7174
URL: https://issues.apache.org/jira/browse/SOLR-7174
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 5.0
Environment: Windows 7. Ubuntu 14.04.
Reporter: Gary Taylor
Assignee: Noble Paul
Labels: dataimportHandler, tika,text-extraction
Attachments: SOLR-7174.patch
Downloaded Solr 5.0.0, on a Windows 7 PC. I ran solr start and then solr
create -c hn2 to create a new core.
I want to index a load of epub files that I've got in a directory. So I
created a data-import.xml (in solr\hn2\conf):
dataConfig
dataSource type=BinFileDataSource name=bin /
document
entity name=files dataSource=null rootEntity=false
processor=FileListEntityProcessor
baseDir=c:/Users/gt/Documents/epub fileName=.*epub
onError=skip
recursive=true
field column=fileAbsolutePath name=id /
field column=fileSize name=size /
field column=fileLastModified name=lastModified /
entity name=documentImport processor=TikaEntityProcessor
url=${files.fileAbsolutePath} format=text
dataSource=bin onError=skip
field column=file name=fileName/
field column=Author name=author meta=true/
field column=title name=title meta=true/
field column=text name=content/
/entity
/entity
/document
/dataConfig
In my solrconfig.xml, I added a requestHandler entry to reference my
data-import.xml:
requestHandler name=/dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=configdata-import.xml/str
/lst
/requestHandler
I renamed managed-schema to schema.xml, and ensured the following doc fields
were setup:
field name=id type=string indexed=true stored=true
required=true multiValued=false /
field name=fileName type=string indexed=true stored=true /
field name=author type=string indexed=true stored=true /
field name=title type=string indexed=true stored=true /
field name=size type=long indexed=true stored=true /
field name=lastModified type=date indexed=true stored=true /
field name=content type=text_en indexed=false stored=true
multiValued=false/
field name=text type=text_en indexed=true stored=false
multiValued=true/
copyField source=content dest=text/
I copied all the jars from dist and contrib\* into server\solr\lib.
Stopping and restarting solr then creates a new managed-schema file and
renames schema.xml to schema.xml.back
All good so far.
Now I go to the web admin for dataimport
(http://localhost:8983/solr/#/hn2/dataimport//dataimport) and try and execute
a full import.
But, the results show Requests: 0, Fetched: 58, Skipped: 0, Processed:1 -
ie. it only adds one document (the very first one) even though it's iterated
over 58!
No errors are reported in the logs.
I can repeat this on Ubuntu 14.04 using the same steps, so it's not Windows
specific.
-
If I change the data-import.xml to use FileDataSource and
PlainTextEntityProcessor and parse txt files, eg:
dataConfig
dataSource type=FileDataSource name=bin /
document
entity name=files dataSource=null rootEntity=false
processor=FileListEntityProcessor
baseDir=c:/Users/gt/Documents/epub fileName=.*txt
field column=fileAbsolutePath name=id /
field column=fileSize name=size /
field column=fileLastModified name=lastModified /
entity name=documentImport
processor=PlainTextEntityProcessor
url=${files.fileAbsolutePath} format=text
dataSource=bin
field column=plainText name=content/
/entity
/entity
/document
/dataConfig
This works. So it's a combo of BinFileDataSource and TikaEntityProcessor
that is failing.
On Windows, I ran Process Monitor, and spotted that only the very first epub
file is actually being read (repeatedly).
With verbose and debug on when running the DIH, I get the following response:
verbose-output: [
entity:files,
[
null,
[jira] [Updated] (LUCENE-6335) fail tests on threadlocal leaks
[
https://issues.apache.org/jira/browse/LUCENE-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-6335:
Attachment: LUCENE-6335.patch
here is a REALLY hacky patch. I based it on tomcat's threadlocal leak detector,
but with some simpler/imperfect logic as a start, since e.g. we aren't wrapping
tests in custom classloader or any of that stuff.
I only fixed TestDemo and LineFileDocs so far and didnt look much more at
failures.
fail tests on threadlocal leaks
---
Key: LUCENE-6335
URL: https://issues.apache.org/jira/browse/LUCENE-6335
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-6335.patch
I know this requires us to do things like close our Analyzers in tests, but I
think we should do it. just experimenting i found other leaks, e.g. in
LineFileDocs.
{noformat}
[junit4] ERROR 0.00s J0 | TestForceMergeForever (suite)
[junit4] Throwable #1: java.lang.IllegalStateException: ThreadLocal
leaks were found:
[junit4] 1.
thread=SUITE-TestForceMergeForever-seed#[AF7141C55A57350E]-worker
value=WeakReferenceHashMap?,Analyzer$TokenStreamComponents
[junit4] 2.
thread=SUITE-TestForceMergeForever-seed#[AF7141C55A57350E]-worker
value=LineFileDocs$DocState
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6335) fail tests on threadlocal leaks
[
https://issues.apache.org/jira/browse/LUCENE-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346470#comment-14346470
]
Robert Muir commented on LUCENE-6335:
-
A lot of the hacky stuff was to get something useful enough to track down the
leak. It makes the code ugly to print
WeakReferenceHashMap?,Analyzer$TokenStreamComponents but this is much
more useful than just java.lang.ref.WeakReference
fail tests on threadlocal leaks
---
Key: LUCENE-6335
URL: https://issues.apache.org/jira/browse/LUCENE-6335
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-6335.patch
I know this requires us to do things like close our Analyzers in tests, but I
think we should do it. just experimenting i found other leaks, e.g. in
LineFileDocs.
{noformat}
[junit4] ERROR 0.00s J0 | TestForceMergeForever (suite)
[junit4] Throwable #1: java.lang.IllegalStateException: ThreadLocal
leaks were found:
[junit4] 1.
thread=SUITE-TestForceMergeForever-seed#[AF7141C55A57350E]-worker
value=WeakReferenceHashMap?,Analyzer$TokenStreamComponents
[junit4] 2.
thread=SUITE-TestForceMergeForever-seed#[AF7141C55A57350E]-worker
value=LineFileDocs$DocState
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6332) join query scanning toField docValue
[
https://issues.apache.org/jira/browse/LUCENE-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev updated LUCENE-6332:
-
Description:
I want to contribute the subj which should do something like ..{{WHERE EXISTS
(SELECT 1 FROM fromSearcher.search(fromQuery) WHERE fromField=toField)}}. It
turns out, that it can be returned by the current method
{{createJoinQuery(...ScoreMode.None)}}
* at first, it should run {{fromQuery}} first, collect {{fromField}} into
{{BytesRefHash}} by {{TermsCollector}}, like it's done now
* then it should return query with _TwoPhase_ Scorer
* which obtains {{toField}} docValue on {{matches()}} and check term for
existence in {{BytesRefHash}}
Do you think it's ever useful? if you do, I can bake a patch.
Anyway, suggest the better API eg separate method, or enum and actual name!
was:
I want to contribute the subj which should do something like ..{{WHERE EXISTS
(SELECT 1 FROM fromSearcher.search(fromQuery) WHERE fromField=toField)}}. It
turns out, that it can be returned by the current method
{{createJoinQuery(...ScoreMode.None)}}
* at first, it should run {{fromQuery}} first, collect {{fromField}} into
{{BytesRefHash}} by {{TermsCollector}}, like it's done now
* then it should return query with _TwoPhase_ Scorer
* which obtains {{toField}} on {{matches()}} and check term for existence in
{{BytesRefHash}}
Do you think it's ever useful? if you do, I can bake a patch.
Anyway, suggest the better API eg separate method, or enum and actual name!
Summary: join query scanning toField docValue (was:
JoinUtil.createSemiJoinQuery())
*ok* referring to _semijoin_ doesn't really make sense, because all Lucene join
queries are semijoins. Hence, I rename the summary, though it still lacks of
clarity.
Any feedback?
join query scanning toField docValue
-
Key: LUCENE-6332
URL: https://issues.apache.org/jira/browse/LUCENE-6332
Project: Lucene - Core
Issue Type: New Feature
Components: modules/join
Affects Versions: 5.0
Reporter: Mikhail Khludnev
I want to contribute the subj which should do something like ..{{WHERE EXISTS
(SELECT 1 FROM fromSearcher.search(fromQuery) WHERE fromField=toField)}}. It
turns out, that it can be returned by the current method
{{createJoinQuery(...ScoreMode.None)}}
* at first, it should run {{fromQuery}} first, collect {{fromField}} into
{{BytesRefHash}} by {{TermsCollector}}, like it's done now
* then it should return query with _TwoPhase_ Scorer
* which obtains {{toField}} docValue on {{matches()}} and check term for
existence in {{BytesRefHash}}
Do you think it's ever useful? if you do, I can bake a patch.
Anyway, suggest the better API eg separate method, or enum and actual name!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7173) Fix ReplicationFactorTest on Windows
[ https://issues.apache.org/jira/browse/SOLR-7173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345986#comment-14345986 ] Mark Miller commented on SOLR-7173: --- Awesome, thank you. I figured we where just missing some retries - this issue should also solve SOLR-6944 (this can also still happen on linux too, just doesn't happen consistently like on windows). Fix ReplicationFactorTest on Windows Key: SOLR-7173 URL: https://issues.apache.org/jira/browse/SOLR-7173 Project: Solr Issue Type: Bug Reporter: Ishan Chattopadhyaya Fix For: 5.1 Attachments: SOLR-7173.patch, SOLR-7173.patch The ReplicationFactorTest fails on the Windows build with NoHttpResponseException, as seen here: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4502/testReport/junit/org.apache.solr.cloud/ReplicationFactorTest/test/ Adding a retry logic similar to HttpPartitionTest's doSend() method makes the test pass on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_31) - Build # 4521 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4521/ Java: 32bit/jdk1.8.0_31 -client -XX:+UseConcMarkSweepGC 1 tests failed. FAILED: org.apache.solr.cloud.ReplicationFactorTest.test Error Message: org.apache.http.NoHttpResponseException: The target server failed to respond Stack Trace: org.apache.solr.client.solrj.SolrServerException: org.apache.http.NoHttpResponseException: The target server failed to respond at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:899) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:752) at org.apache.solr.cloud.ReplicationFactorTest.testRf3(ReplicationFactorTest.java:284) at org.apache.solr.cloud.ReplicationFactorTest.test(ReplicationFactorTest.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:950) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:925) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at
[jira] [Updated] (LUCENE-6334) Term Vector Highlighter does not properly span neighboring term offsets
[
https://issues.apache.org/jira/browse/LUCENE-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Earle updated LUCENE-6334:
Description:
If you are using term vectors for fast vector highlighting along with a
multivalue field while matching a phrase that crosses two elements, then it
will not properly highlight even though it _properly_ finds the correct values
to highlight.
A good example of this is when matching source code, where you might have lines
like:
{code}
one two three five
two three four
five six five
six seven eight nine eight nine eight nine eight nine eight nine eight nine
eight nine
ten eleven
twelve thirteen
{code}
Matching the phrase four five will return
{code}
two three four
five six five
six seven eight nine eight nine eight nine eight nine eight
eight nine
ten eleven
{code}
However, it does not properly highlight four (on the first line) and five
(on the second line) _and_ it is returning too many lines, but not all of them.
The problem lies in the [BaseFragmentsBuilder at line 269|
https://github.com/apache/lucene-solr/blob/trunk/lucene/highlighter/src/java/org/apache/lucene/search/vectorhighlight/BaseFragmentsBuilder.java#L269]
because it is not checking for cross-coverage:
{code}
boolean started = toffs.getStartOffset() = fieldStart;
boolean ended = toffs.getEndOffset() = fieldEnd;
// existing behavior:
if (started ended) {
toffsList.add(toffs);
toffsIterator.remove();
}
else if (started) {
toffsList.add(new Toffs(toffs.getStartOffset(), field.end));
// toffsIterator.remove(); // is this necessary?
}
else if (ended) {
toffsList.add(new Toffs(fieldStart, toff.getEndOffset()));
// toffsIterator.remove(); // is this necessary?
}
else if (toffs.getEndOffset() fieldEnd) {
// ie the toff spans whole field
toffsList.add(new Toffs(fieldStart, fieldEnd));
// toffsIterator.remove(); // is this necessary?
}
{code}
was:
If you are using term vectors for fast vector highlighting along with a
multivalue field while matching a phrase that crosses two elements, then it
will not properly highlight even though it _properly_ finds the correct values
to highlight.
A good example of this is when matching source code, where you might have lines
like:
{code}
one two three five
two three four
five six five
six seven eight nine eight nine eight nine eight nine eight nine eight nine
eight nine
ten eleven
twelve thirteen
{code}
Matching the phrase four five will return
{code}
two three four
five six five
six seven eight nine eight nine eight nine eight nine eight
eight nine
ten eleven
{code}
However, it does not properly highlight four (on the first line) and five
(on the second line).
The problem lies in the [BaseFragmentsBuilder at line 269|
https://github.com/apache/lucene-solr/blob/trunk/lucene/highlighter/src/java/org/apache/lucene/search/vectorhighlight/BaseFragmentsBuilder.java#L269]
because it is not checking for cross-coverage:
{code}
boolean started = toffs.getStartOffset() = fieldStart;
boolean ended = toffs.getEndOffset() = fieldEnd;
// existing behavior:
if (started ended) {
toffsList.add(toffs);
toffsIterator.remove();
}
else if (started) {
toffsList.add(new Toffs(toffs.getStartOffset(), field.end));
// toffsIterator.remove(); // is this necessary?
}
else if (ended) {
toffsList.add(new Toffs(fieldStart, toff.getEndOffset()));
// toffsIterator.remove(); // is this necessary?
}
else if (toffs.getEndOffset() fieldEnd) {
// ie the toff spans whole field
toffsList.add(new Toffs(fieldStart, fieldEnd));
// toffsIterator.remove(); // is this necessary?
}
{code}
Term Vector Highlighter does not properly span neighboring term offsets
---
Key: LUCENE-6334
URL: https://issues.apache.org/jira/browse/LUCENE-6334
Project: Lucene - Core
Issue Type: Bug
Components: core/termvectors, modules/highlighter
Reporter: Chris Earle
Labels: easyfix
If you are using term vectors for fast vector highlighting along with a
multivalue field while matching a phrase that crosses two elements, then it
will not properly highlight even though it _properly_ finds the correct
values to highlight.
A good example of this is when matching source code, where you might have
lines like:
{code}
one two three five
two three four
five six five
six seven eight nine eight nine eight nine eight nine eight nine eight nine
eight nine
ten eleven
twelve thirteen
{code}
Matching the phrase four five will return
{code}
two three four
five six five
six seven eight nine eight nine eight nine eight nine eight
eight nine
ten eleven
{code}
However, it does not properly highlight four (on the first line) and five
(on the second line) _and_ it is returning too many lines, but not
[jira] [Updated] (SOLR-7187) SolrCloud does not fully clean collection after delete
[
https://issues.apache.org/jira/browse/SOLR-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated SOLR-7187:
Description:
When I attempt to delete a collection using
{{/admin/collections?action=DELETEname=collection1}} if I go into HDFS I will
still see remnants from the collection. No files, but empty directories stick
around.
{noformat}
[root@solr1 ~]# sudo -u hdfs hdfs dfs -ls -R /solr/collection1
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node1
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node2
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node3
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node4
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node5
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node6
{noformat}
(Edit: I had the wrong log portion here originally)
In the logs, after deleting all the data, I see:
{noformat}
2015-03-03 16:15:14,762 INFO org.apache.solr.servlet.SolrDispatchFilter:
[admin] webapp=null path=/admin/cores
params={deleteInstanceDir=trueaction=UNLOADcore=collection1_shard5_replica1wt=javabinqt=/admin/coresdeleteDataDir=trueversion=2}
status=0 QTime=362
2015-03-03 16:15:14,787 INFO org.apache.solr.common.cloud.ZkStateReader: A
cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
2015-03-03 16:15:14,854 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:14,879 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:14,896 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:14,920 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:15,151 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:15,170 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
2015-03-03 16:15:15,279 INFO org.apache.solr.common.cloud.ZkStateReader: A
cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
2015-03-03 16:15:15,546 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/collection-queue-work/qnr-16
state: SyncConnected type NodeDataChanged
2015-03-03 16:15:15,562 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/collection-queue-work state:
SyncConnected type NodeChildrenChanged
2015-03-03 16:15:15,562 INFO org.apache.solr.cloud.OverseerCollectionProcessor:
Overseer Collection Processor: Message
id:/overseer/collection-queue-work/qn-16 complete,
response:{success={solr1.example.com:8983_solr={responseHeader={status=0,QTime=207}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=243}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=243}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=342}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=346}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=362
{noformat}
This might be related to SOLR-5023, but I'm not sure.
was:
When I attempt to delete a collection using
{{/admin/collections?action=DELETEname=collection1}} if I go into HDFS I will
still see remnants from the collection. No files, but empty directories stick
around.
{noformat}
[root@drob-search-cdh-25682 ~]# sudo -u hdfs hdfs dfs -ls -R /solr/collection1
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node1
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node2
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node3
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node4
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node5
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node6
{noformat}
In the logs, after deleting all the data, I see:
{noformat}
2015-03-03 15:42:44,453 INFO org.apache.solr.handler.admin.CollectionsHandler:
Deleting Collection : name=collection1action=DELETE
2015-03-03 15:42:44,465 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path:
[jira] [Assigned] (SOLR-6275) Improve accuracy of QTime reporting
[
https://issues.apache.org/jira/browse/SOLR-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramkumar Aiyengar reassigned SOLR-6275:
---
Assignee: Ramkumar Aiyengar (was: Mark Miller)
Improve accuracy of QTime reporting
---
Key: SOLR-6275
URL: https://issues.apache.org/jira/browse/SOLR-6275
Project: Solr
Issue Type: Improvement
Components: search
Reporter: Ramkumar Aiyengar
Assignee: Ramkumar Aiyengar
Priority: Minor
Attachments: SOLR-6275.patch
Currently, {{QTime}} uses {{currentTimeMillis}} instead of {{nanoTime}} and
hence is not suitable for time measurements. Further, it is really started
after all the dispatch logic in {{SolrDispatchFilter}} (same with the top
level timing reported by {{debug=timing}}) which may or may not be expensive,
and hence may not fully represent the time taken by the search. This is to
remedy both cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6275) Improve accuracy of QTime reporting
[
https://issues.apache.org/jira/browse/SOLR-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramkumar Aiyengar updated SOLR-6275:
Attachment: SOLR-6275.patch
Updated patch, passes tests and precommit.
Improve accuracy of QTime reporting
---
Key: SOLR-6275
URL: https://issues.apache.org/jira/browse/SOLR-6275
Project: Solr
Issue Type: Improvement
Components: search
Reporter: Ramkumar Aiyengar
Assignee: Mark Miller
Priority: Minor
Attachments: SOLR-6275.patch
Currently, {{QTime}} uses {{currentTimeMillis}} instead of {{nanoTime}} and
hence is not suitable for time measurements. Further, it is really started
after all the dispatch logic in {{SolrDispatchFilter}} (same with the top
level timing reported by {{debug=timing}}) which may or may not be expensive,
and hence may not fully represent the time taken by the search. This is to
remedy both cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7177) ConcurrentUpdateSolrClient should log connection information on http failures
[
https://issues.apache.org/jira/browse/SOLR-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346092#comment-14346092
]
Mark Miller commented on SOLR-7177:
---
Yeah, I think thats fine.
ConcurrentUpdateSolrClient should log connection information on http failures
--
Key: SOLR-7177
URL: https://issues.apache.org/jira/browse/SOLR-7177
Project: Solr
Issue Type: Improvement
Affects Versions: 4.10.3, 5.0
Reporter: Vamsee Yarlagadda
Priority: Minor
Attachments: SOLR-7177.patch, SOLR-7177v2.patch
I notice when there is an http connection failure, we simply log the error
but not the connection information. It would be good to log this info to make
debugging easier.
e.g:
1.
{code}
2015-02-27 08:56:51,503 ERROR org.apache.solr.update.StreamingSolrServers:
error
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:235)
{code}
2.
{code}
2015-02-27 10:26:12,363 ERROR org.apache.solr.update.StreamingSolrServers:
error
org.apache.http.NoHttpResponseException: The target server failed to respond
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:235)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
As we can notice, we can see the
[jira] [Commented] (SOLR-7173) Fix ReplicationFactorTest on Windows
[ https://issues.apache.org/jira/browse/SOLR-7173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345988#comment-14345988 ] Mark Miller commented on SOLR-7173: --- bq. should I update that code as well with this iterative/recursive retry? Yes. All the better if we can just consolidate the code in some test util method rather than duping it. Fix ReplicationFactorTest on Windows Key: SOLR-7173 URL: https://issues.apache.org/jira/browse/SOLR-7173 Project: Solr Issue Type: Bug Reporter: Ishan Chattopadhyaya Fix For: 5.1 Attachments: SOLR-7173.patch, SOLR-7173.patch The ReplicationFactorTest fails on the Windows build with NoHttpResponseException, as seen here: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4502/testReport/junit/org.apache.solr.cloud/ReplicationFactorTest/test/ Adding a retry logic similar to HttpPartitionTest's doSend() method makes the test pass on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7177) ConcurrentUpdateSolrClient should log connection information on http failures
[
https://issues.apache.org/jira/browse/SOLR-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346028#comment-14346028
]
Vamsee Yarlagadda commented on SOLR-7177:
-
bq. In that case you know the URL - you had to init the class with it and this
is part of your client code. You can log whatever you want if the call fails on
the client side.
True. But at the same time we would have to add this logic of adding url
information at multiple places right (one at the server, one at the solrj
client)? If this is considered as the most basic info to be attached to the
exception we can do it at the ConcurrentUpdateSolrClient. If we need to add
some extra information on top of adding url info then this extra work can be
done at the place where we are init'ing the class. Thoughts?
ConcurrentUpdateSolrClient should log connection information on http failures
--
Key: SOLR-7177
URL: https://issues.apache.org/jira/browse/SOLR-7177
Project: Solr
Issue Type: Improvement
Affects Versions: 4.10.3, 5.0
Reporter: Vamsee Yarlagadda
Priority: Minor
Attachments: SOLR-7177.patch, SOLR-7177v2.patch
I notice when there is an http connection failure, we simply log the error
but not the connection information. It would be good to log this info to make
debugging easier.
e.g:
1.
{code}
2015-02-27 08:56:51,503 ERROR org.apache.solr.update.StreamingSolrServers:
error
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:235)
{code}
2.
{code}
2015-02-27 10:26:12,363 ERROR org.apache.solr.update.StreamingSolrServers:
error
org.apache.http.NoHttpResponseException: The target server failed to respond
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
at
[jira] [Commented] (SOLR-7121) Solr nodes should go down based on configurable thresholds and not rely on resource exhaustion
[ https://issues.apache.org/jira/browse/SOLR-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345973#comment-14345973 ] Mark Miller commented on SOLR-7121: --- Cool, glad to see the progress on this. bq. GC-time test will need to run for quite sometime before it can detect the same I was thinking that probably we want some base class that can gather all these types of stats - then we can have a mock one that returns whatever we want for tests, and a real impl that returns legit values. Solr nodes should go down based on configurable thresholds and not rely on resource exhaustion -- Key: SOLR-7121 URL: https://issues.apache.org/jira/browse/SOLR-7121 Project: Solr Issue Type: New Feature Reporter: Sachin Goyal Attachments: SOLR-7121.patch, SOLR-7121.patch, SOLR-7121.patch, SOLR-7121.patch, SOLR-7121.patch Currently, there is no way to control when a Solr node goes down. If the server is having high GC pauses or too many threads or is just getting too many queries due to some bad load-balancer, the cores in the machine keep on serving unless they exhaust the machine's resources and everything comes to a stall. Such a slow-dying core can affect other cores as well by taking huge time to serve their distributed queries. There should be a way to specify some threshold values beyond which the targeted core can its ill-health and proactively go down to recover. When the load improves, the core should come up automatically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6275) Improve accuracy of QTime reporting
[
https://issues.apache.org/jira/browse/SOLR-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346002#comment-14346002
]
ASF subversion and git services commented on SOLR-6275:
---
Commit 1663829 from andyetitmo...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1663829 ]
SOLR-6275: Improve accuracy of QTime reporting
Improve accuracy of QTime reporting
---
Key: SOLR-6275
URL: https://issues.apache.org/jira/browse/SOLR-6275
Project: Solr
Issue Type: Improvement
Components: search
Reporter: Ramkumar Aiyengar
Assignee: Ramkumar Aiyengar
Priority: Minor
Attachments: SOLR-6275.patch
Currently, {{QTime}} uses {{currentTimeMillis}} instead of {{nanoTime}} and
hence is not suitable for time measurements. Further, it is really started
after all the dispatch logic in {{SolrDispatchFilter}} (same with the top
level timing reported by {{debug=timing}}) which may or may not be expensive,
and hence may not fully represent the time taken by the search. This is to
remedy both cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6275) Improve accuracy of QTime reporting
[
https://issues.apache.org/jira/browse/SOLR-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345961#comment-14345961
]
Mark Miller commented on SOLR-6275:
---
+1, looks okay to me.
Improve accuracy of QTime reporting
---
Key: SOLR-6275
URL: https://issues.apache.org/jira/browse/SOLR-6275
Project: Solr
Issue Type: Improvement
Components: search
Reporter: Ramkumar Aiyengar
Assignee: Ramkumar Aiyengar
Priority: Minor
Attachments: SOLR-6275.patch
Currently, {{QTime}} uses {{currentTimeMillis}} instead of {{nanoTime}} and
hence is not suitable for time measurements. Further, it is really started
after all the dispatch logic in {{SolrDispatchFilter}} (same with the top
level timing reported by {{debug=timing}}) which may or may not be expensive,
and hence may not fully represent the time taken by the search. This is to
remedy both cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: reuseAddress default in Solr jetty.xml
But all too often necessary :) On Tue, Mar 3, 2015 at 12:14 AM Ramkumar R. Aiyengar andyetitmo...@gmail.com wrote: I agree, sigkill is typically the last resort.. On 3 Mar 2015 00:49, Reitzel, Charles charles.reit...@tiaa-cref.org wrote: My bad. Too long away from sockets since cleaning up those shutdown handlers. Your point is well taken, on the server side the risks of consuming a stray echo packet are fairly low (but non-zero, if you’ve ever spent any quality time with tcpdump/wireshark). Still, in a production setting, SIGKILL (aka “kill -9”) should be a last resort after more reasonable methods (e.g. SIGINT, SIGTERM, SIGSTOP) have failed. *From:* Ramkumar R. Aiyengar [mailto:andyetitmo...@gmail.com] *Sent:* Monday, March 02, 2015 7:00 PM *To:* dev@lucene.apache.org *Subject:* RE: reuseAddress default in Solr jetty.xml No, reuseAddress doesn't allow you to have two processes, old and new, listen to the same port. There's no option which allows you to do that. Tl;DR This can happen when you have a connection to a server which gets killed hard and comes back up immediately So here's what happens. When a server normally shuts down, it triggers an active close on all open TCP connections it has. That sends a three way msg exchange with the remote recipient (FIN, FIN+ACK, ACK) at the end of which the socket is closed and the kernel puts it in a TIME_WAIT state for a few minutes in the background (depends on the OS, maximum tends to be 4 mins). This is needed to allow for reordered older packets to reach the machine just in case. Now typically if the server restarts within that period and tries to bind again to the same port, the kernel is smart enough to not complain that there is an existing socket in TIME_WAIT, because it knows the last sequence number it used for the final message in the previous process, and since sequence numbers are always increasing, it can reject any messages before that sequence number as a new process has now taken the port. Trouble is with abnormal shutdown. There's no time for a proper goodbye, so the kernel marks the socket to respond to remote packets with a rude RST (reset). Since there has been no goodbye with the remote end, it also doesn't know the last sequence number to delineate if a new process binds to the same port. Hence by default it denies binding to the new port for the TIME_WAIT period to avoid the off chance a stray packet gets picked up by the new process and utterly confuses it. By setting reuseAddress, you are essentially waiving off this protection. Note that this possibility of confusion is unbelievably miniscule in the first place (both the source and destination host:port should be the same and the client port is generally randomly allocated). If the port we are talking of is a local port, it's almost impossible -- you have bigger problems if a TCP packet is lost or delayed within the same machine! As to Shawn's point, for Solr's stop port, you essentially need to be trying to actively shutdown the server using the stop port, or be within a few minutes of such an attempt while the server is killed. Just the server being killed without any active connection to it is not going to cause this issue. Hi Ram, It appears the problem is that the old solr/jetty process is actually still running when the new solr/jetty process is started. That’s the problem that needs fixing. This is not a rare problem in systems with worker threads dedicated to different tasks. These threads need to wake up in response to the shutdown signal/command, as well the normal inputs. It’s a bug I’ve created and fixed a couple times over the years … :-) I wouldn’t know where to start with Solr. But, as I say, re-using the port is a band-aid. I’ve yet to see a case where it is the best solution. best, Charlie *From:* Ramkumar R. Aiyengar [mailto:andyetitmo...@gmail.com] *Sent:* Saturday, February 28, 2015 8:15 PM *To:* dev@lucene.apache.org *Subject:* Re: reuseAddress default in Solr jetty.xml Hey Charles, see my explanation above on why this is needed. If Solr has to be killed, it would generally be immediately restarted. This would normally not the case, except when things are potentially misconfigured or if there is a bug, but not doing so makes the impact worse.. In any case, turns out really that reuseAddress is true by default for the connectors we use, so that really isn't the issue. The issue more specifically is that the stop port doesn't do it, so the actual port by itself starts just fine on a restart, but the stop port fails to bind -- and there's no way currently in Jetty to configure that. Based on my question in the jetty mailing list, I have now created an issue for them.. https://bugs.eclipse.org/bugs/show_bug.cgi?id=461133 On Fri, Feb 27, 2015 at 3:03 PM, Reitzel, Charles charles.reit...@tiaa-cref.org wrote: Disclaimer: I’m not a Solr
Re: Starting Solr 5 with -e cloud -- how to restart?
Something like: bin/solr start -c -z localhost:2181 -p 8982 -s example/cloud/node2/solr But note that you probably have to run this N times where N is the number of servers you specified when you created the initial setup. Best, Erick On Tue, Mar 3, 2015 at 6:21 AM, Shawn Heisey apa...@elyograg.org wrote: I've been experimenting with the new start script in 5.0, and have come across something I can't figure out. It's easy enough to start up a SolrCloud example (bin/solr start -e cloud, answer the prompts) ... but once I've done that, how do I restart it without telling it how to build the cloud example again? I can stop it easily enough with bin/solr stop -all ... but if I then do bin/solr start it only starts one Solr instance. Any ideas would be appreciated. Thanks, Shawn - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7187) SolrCloud does not fully clean collection after delete
Mike Drob created SOLR-7187:
---
Summary: SolrCloud does not fully clean collection after delete
Key: SOLR-7187
URL: https://issues.apache.org/jira/browse/SOLR-7187
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.10.3
Reporter: Mike Drob
When I attempt to delete a collection using
{{/admin/collections?action=DELETEname=collection1}} if I go into HDFS I will
still see remnants from the collection. No files, but empty directories stick
around.
{noformat}
[root@drob-search-cdh-25682 ~]# sudo -u hdfs hdfs dfs -ls -R /solr/collection1
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node1
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node2
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node3
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node4
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node5
drwxr-xr-x - solr solr 0 2015-03-03 15:42
/solr/collection1/core_node6
{noformat}
In the logs, after deleting all the data, I see:
{noformat}
2015-03-03 15:42:44,453 INFO org.apache.solr.handler.admin.CollectionsHandler:
Deleting Collection : name=collection1action=DELETE
2015-03-03 15:42:44,465 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/collection-queue-work state:
SyncConnected type NodeChildrenChanged
2015-03-03 15:42:44,474 INFO org.apache.solr.cloud.OverseerCollectionProcessor:
Overseer Collection Processor: Get the message
id:/overseer/collection-queue-work/qn-12 message:{
operation:deletecollection,
name:collection1}
2015-03-03 15:42:44,474 WARN org.apache.solr.cloud.OverseerCollectionProcessor:
OverseerCollectionProcessor.processMessage : deletecollection , {
operation:deletecollection,
name:collection1}
2015-03-03 15:42:44,474 INFO org.apache.solr.cloud.OverseerCollectionProcessor:
Executing Collection Cmd :
action=UNLOADdeleteInstanceDir=truedeleteDataDir=true
2015-03-03 15:42:44,475 ERROR
org.apache.solr.cloud.OverseerCollectionProcessor: Collection deletecollection
of deletecollection failed:org.apache.solr.common.SolrException: Could not find
collection:collection1
at
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:145)
at
org.apache.solr.cloud.OverseerCollectionProcessor.collectionCmd(OverseerCollectionProcessor.java:1074)
at
org.apache.solr.cloud.OverseerCollectionProcessor.deleteCollection(OverseerCollectionProcessor.java:296)
at
org.apache.solr.cloud.OverseerCollectionProcessor.processMessage(OverseerCollectionProcessor.java:244)
at
org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:171)
at java.lang.Thread.run(Thread.java:745)
2015-03-03 15:42:44,498 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/collection-queue-work/qnr-12
state: SyncConnected type NodeDataChanged
2015-03-03 15:42:44,506 INFO org.apache.solr.cloud.DistributedQueue:
LatchChildWatcher fired on path: /overseer/collection-queue-work state:
SyncConnected type NodeChildrenChanged
2015-03-03 15:42:44,506 INFO org.apache.solr.cloud.OverseerCollectionProcessor:
Overseer Collection Processor: Message
id:/overseer/collection-queue-work/qn-12 complete, response:{Operation
deletecollection caused exception:=org.apache.solr.common.SolrException: Could
not find collection:collection1,exception={msg=Could not find
collection:collection1,rspCode=400}}
{noformat}
This might be related to SOLR-5023, but I'm not sure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6334) Term Vector Highlighter does not properly span neighboring term offsets
Chris Earle created LUCENE-6334:
---
Summary: Term Vector Highlighter does not properly span
neighboring term offsets
Key: LUCENE-6334
URL: https://issues.apache.org/jira/browse/LUCENE-6334
Project: Lucene - Core
Issue Type: Bug
Components: core/termvectors, modules/highlighter
Reporter: Chris Earle
If you are using term vectors for fast vector highlighting along with a
multivalue field while matching a phrase that crosses two elements, then it
will not properly highlight even though it _properly_ finds the correct values
to highlight.
A good example of this is when matching source code, where you might have lines
like:
{code}
one two three five
two three four
five six five
six seven eight nine eight nine eight nine eight nine eight nine eight nine
eight nine
ten eleven
twelve thirteen
{code}
Matching the phrase four five will return
{code}
two three four
five six five
six seven eight nine eight nine eight nine eight nine eight
eight nine
ten eleven
{code}
However, it does not properly highlight four (on the first line) and five
(on the second line).
The problem lies in the [BaseFragmentsBuilder at line 269|
https://github.com/apache/lucene-solr/blob/trunk/lucene/highlighter/src/java/org/apache/lucene/search/vectorhighlight/BaseFragmentsBuilder.java#L269]
because it is not checking for cross-coverage:
{code}
boolean started = toffs.getStartOffset() = fieldStart;
boolean ended = toffs.getEndOffset() = fieldEnd;
// existing behavior:
if (started ended) {
toffsList.add(toffs);
toffsIterator.remove();
}
else if (started) {
toffsList.add(new Toffs(toffs.getStartOffset(), field.end));
// toffsIterator.remove(); // is this necessary?
}
else if (ended) {
toffsList.add(new Toffs(fieldStart, toff.getEndOffset()));
// toffsIterator.remove(); // is this necessary?
}
else if (toffs.getEndOffset() fieldEnd) {
// ie the toff spans whole field
toffsList.add(new Toffs(fieldStart, fieldEnd));
// toffsIterator.remove(); // is this necessary?
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2710 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2710/ 6 tests failed. REGRESSION: org.apache.solr.handler.component.DistributedMLTComponentTest.test Error Message: Timeout occured while waiting response from server at: http://127.0.0.1:46014//collection1 Stack Trace: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:46014//collection1 at __randomizedtesting.SeedInfo.seed([51DE0205A514CA70:D98A3DDF0BE8A788]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:568) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:214) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:210) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:131) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:309) at org.apache.solr.BaseDistributedSearchTestCase.queryServer(BaseDistributedSearchTestCase.java:548) at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:596) at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:578) at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:557) at org.apache.solr.handler.component.DistributedMLTComponentTest.test(DistributedMLTComponentTest.java:126) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:950) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:925) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
