[jira] [Updated] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that maven dependencies' scope can be specified appropriately, and the derived Maven dependencie
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated SOLR-6006: - Summary: Separate test and compile scope dependencies in the Solrj ivy.xml file, so that maven dependencies' scope can be specified appropriately, and the derived Maven dependencies get filled out properly in the Solrj POM (was: Separate test and compile scope dependencies in ivy.xml files, so that maven dependencies' scope can be specified appropriately ) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that maven dependencies' scope can be specified appropriately, and the derived Maven dependencies get filled out properly in the Solrj POM Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Priority: Minor Attachments: SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated SOLR-6006: - Summary: Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM (was: Separate test and compile scope dependencies in the Solrj ivy.xml file, so that maven dependencies' scope can be specified appropriately, and the derived Maven dependencies get filled out properly in the Solrj POM ) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM --- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Priority: Minor Attachments: SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980731#comment-13980731 ] ASF subversion and git services commented on SOLR-6006: --- Commit 1589937 from [~steve_rowe] in branch 'dev/trunk' [ https://svn.apache.org/r1589937 ] SOLR-6006: Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM --- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Priority: Minor Attachments: SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980734#comment-13980734 ] ASF subversion and git services commented on SOLR-6006: --- Commit 1589940 from [~steve_rowe] in branch 'dev/trunk' [ https://svn.apache.org/r1589940 ] SOLR-6006: IntelliJ config for solrj/test-lib/ Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM --- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Priority: Minor Attachments: SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
maxThreads in Jetty
In the /etc/jetty.xml delivered with Solr, maxThreads is set to 10,000. I wonder why maxThreads for Jetty with Solr is so high? The default case for a Solr instance is that processing is isolated to machines controlled by the owner: Performance is dictated by local CPU, memory storage latency. There is no waiting for external services. With this in mind, I would not expect throughput to rise after a certain number of concurrent searches: When CPUs are maxed and storage is maxed, starting new Threads just increases processing time for the already running ones. A fairly beefy machine nowadays might be 24 Hyper-Threaded cores with SSDs in RAID 0 as backend. Let's say that CPU is the bottleneck here. We multiply with 2 (way too much) for the Hyper-Threading and another 2 to compensate for locks. Back of the envelope says that more than 100 concurrent searches on that machine will not increase throughput. Fewer CPUs or slower storage would only lower that number. Cache hits is a joker here as it takes very little CPU, but it would take a cache hit rate of 99% to get increased throughput from the 10,000 threads. Setting maxThread too high is bad, according to http://67-23-9-112.static.slicehost.net/doc/optimization.html The obvious problem for Solr is memory usage as some searches require a non-trivial amount of temporary heap space; notably faceting with a bitset for the hits and structures for counting (int[] or HashMap, depending on implementation). A modest index of 5M documents with 100K unique values in a facet field (used with fc or DocValues) takes 1MByte+ of memory for a single search or 10GByte+ with 10,000. The potentially huge discrepancy between the memory requirements under light load vs. heavy load seems like a trap. An unplanned burst of request might very well bring the installation down. Alternatively one could over-allocate memory to guard against OOM, but since throughput does not increase past a given amount of concurrent searches (where a given amount by my logic is far less than 10,000), this is essentially wasted: Queueing would give the same throughput with lower memory requirements. By the logic above, maxThreads of 100 or maybe 200 would be an appropriate default for Jetty with Solr. So why the 10,000? - Toke Eskildsen, State and Univeristy Library, Denmark - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [VOTE] Lucene/Solr 4.8.0 RC2
Here is my +1: SUCCESS! [1:53:39.982427] Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Thursday, April 24, 2014 11:54 PM To: dev@lucene.apache.org Subject: [VOTE] Lucene/Solr 4.8.0 RC2 Hi, I prepared a second release candidate of Lucene and Solr 4.8.0. The artifacts can be found here: = http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0- RC2-rev1589874/ This RC contains the additional fixes for SOLR-6011, LUCENE-5626, and LUCENE-5630. Please check the artifacts and give your vote in the next 72 hrs. Uwe P.S.: Here's my smoker command line: $ JAVA_HOME=$HOME/jdk1.7.0_55 JAVA7_HOME=$HOME/jdk1.7.0_55 python3.2 -u smokeTestRelease.py ' http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-RC2- rev1589874/' 1589874 4.8.0 tmp - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Lucene/Solr 4.8.0 RC2
+1 SUCCESS! [1:46:42.182320] Ahmet On Friday, April 25, 2014 9:55 AM, Uwe Schindler u...@thetaphi.de wrote: Here is my +1: SUCCESS! [1:53:39.982427] Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Thursday, April 24, 2014 11:54 PM To: dev@lucene.apache.org Subject: [VOTE] Lucene/Solr 4.8.0 RC2 Hi, I prepared a second release candidate of Lucene and Solr 4.8.0. The artifacts can be found here: = http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0- RC2-rev1589874/ This RC contains the additional fixes for SOLR-6011, LUCENE-5626, and LUCENE-5630. Please check the artifacts and give your vote in the next 72 hrs. Uwe P.S.: Here's my smoker command line: $ JAVA_HOME=$HOME/jdk1.7.0_55 JAVA7_HOME=$HOME/jdk1.7.0_55 python3.2 -u smokeTestRelease.py ' http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-RC2- rev1589874/' 1589874 4.8.0 tmp - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4478) Allow cores to specify a named config set in non-SolrCloud mode
[ https://issues.apache.org/jira/browse/SOLR-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980787#comment-13980787 ] Alan Woodward commented on SOLR-4478: - Good catch! I'll update the documentation, but really we ought not to care about casing. Allow cores to specify a named config set in non-SolrCloud mode --- Key: SOLR-4478 URL: https://issues.apache.org/jira/browse/SOLR-4478 Project: Solr Issue Type: Improvement Affects Versions: 4.2, 5.0 Reporter: Erick Erickson Assignee: Alan Woodward Fix For: 4.8, 5.0 Attachments: SOLR-4478-take2.patch, SOLR-4478-take2.patch, SOLR-4478-take2.patch, SOLR-4478-take2.patch, SOLR-4478.patch, SOLR-4478.patch, solr.log Part of moving forward to the new way, after SOLR-4196 etc... I propose an additional parameter specified on the core node in solr.xml or as a parameter in the discovery mode core.properties file, call it configSet, where the value provided is a path to a directory, either absolute or relative. Really, this is as though you copied the conf directory somewhere to be used by more than one core. Straw-man: There will be a directory solr_home/configsets which will be the default. If the configSet parameter is, say, myconf, then I'd expect a directory named myconf to exist in solr_home/configsets, which would look something like solr_home/configsets/myconf/schema.xml solrconfig.xml stopwords.txt velocity velocity/query.vm etc. If multiple cores used the same configSet, schema, solrconfig etc. would all be shared (i.e. shareSchema=true would be assumed). I don't see a good use-case for _not_ sharing schemas, so I don't propose to allow this to be turned off. Hmmm, what if shareSchema is explicitly set to false in the solr.xml or properties file? I'd guess it should be honored but maybe log a warning? Mostly I'm putting this up for comments. I know that there are already thoughts about how this all should work floating around, so before I start any work on this I thought I'd at least get an idea of whether this is the way people are thinking about going. Configset can be either a relative or absolute path, if relative it's assumed to be relative to solr_home. Thoughts? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Early Access builds for JDK 9 b09, JDK 8u20 b10 and JDK 7U60 b15 are available on java.net
Hi Uwe,Dawid, Early Access builds for JDK 9 b09 https://jdk9.java.net/download/, JDK 8u20 b10 https://jdk8.java.net/download.html and JDK 7U60 b15 https://jdk7.java.net/download.html are available on java.net. As we enter the later phases of development for JDK 7u60 JDK 8u20 , please log any show stoppers as soon as possible. Rgds, Rory -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b05) - Build # 10156 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10156/ Java: 64bit/jdk1.8.0_20-ea-b05 -XX:-UseCompressedOops -XX:+UseParallelGC 4 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.analysis.TestFoldingMultitermExtrasQuery Error Message: org/apache/commons/logging/LogFactory Stack Trace: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0) at org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60) at org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271) at org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127) at org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:155) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:205) at org.apache.solr.util.TestHarness.init(TestHarness.java:137) at org.apache.solr.util.TestHarness.init(TestHarness.java:147) at org.apache.solr.util.TestHarness.init(TestHarness.java:98) at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:569) at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:559) at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:379) at org.apache.solr.analysis.TestFoldingMultitermExtrasQuery.beforeTests(TestFoldingMultitermExtrasQuery.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:744) FAILED: junit.framework.TestSuite.org.apache.solr.schema.TestICUCollationField Error Message: org/apache/commons/logging/LogFactory Stack Trace: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0) at org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60) at org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271) at org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127) at org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117) at
Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b05) - Build # 10156 - Failure!
I’m working on a fix. - Steve On Apr 25, 2014, at 4:25 AM, Policeman Jenkins Server jenk...@thetaphi.de wrote: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10156/ Java: 64bit/jdk1.8.0_20-ea-b05 -XX:-UseCompressedOops -XX:+UseParallelGC 4 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.analysis.TestFoldingMultitermExtrasQuery Error Message: org/apache/commons/logging/LogFactory Stack Trace: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0) at org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60) at org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271) at org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127) at org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:155) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:205) at org.apache.solr.util.TestHarness.init(TestHarness.java:137) at org.apache.solr.util.TestHarness.init(TestHarness.java:147) at org.apache.solr.util.TestHarness.init(TestHarness.java:98) at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:569) at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:559) at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:379) at org.apache.solr.analysis.TestFoldingMultitermExtrasQuery.beforeTests(TestFoldingMultitermExtrasQuery.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:744) FAILED: junit.framework.TestSuite.org.apache.solr.schema.TestICUCollationField Error Message: org/apache/commons/logging/LogFactory Stack Trace: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0) at org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60) at org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271) at org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127) at org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116) at
[jira] [Updated] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated SOLR-6006: - Attachment: SOLR-6006-contribs.patch I neglected to test the Solr contribs, and they were failing after I committed the patch on trunk because they all have a test-scope dependency on jcl-over-slf4j - Uwe's Jenkins has made its unhappiness known: [http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10156/] This patch fixes the build by giving each Solr contrib a {{test-lib/}} if it doesn't already have it, and adding jcl-over-slf4j to its test scope dependencies. Committing shortly. Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM --- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Priority: Minor Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980813#comment-13980813 ] ASF subversion and git services commented on SOLR-6006: --- Commit 1589953 from [~steve_rowe] in branch 'dev/trunk' [ https://svn.apache.org/r1589953 ] SOLR-6006: fix Solr contrib test dependencies by adding jcl-over-slf4j and retrieving it into each contrib's test-lib/ directory Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM --- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Priority: Minor Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6006) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the correspondi
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe updated SOLR-6006: - Summary: Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the corresponding POMs (was: Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the corresponding POMs -- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Priority: Minor Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the correspon
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980832#comment-13980832 ] ASF subversion and git services commented on SOLR-6006: --- Commit 1589959 from [~steve_rowe] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1589959 ] SOLR-6006: Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the corresponding POMs (merged trunk r1589937, r1589940, and r158953) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the corresponding POMs -- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Priority: Minor Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b05) - Build # 10156 - Failure!
I committed a fix. - Steve On Apr 25, 2014, at 4:27 AM, Steve Rowe sar...@gmail.com wrote: I’m working on a fix. - Steve On Apr 25, 2014, at 4:25 AM, Policeman Jenkins Server jenk...@thetaphi.de wrote: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10156/ Java: 64bit/jdk1.8.0_20-ea-b05 -XX:-UseCompressedOops -XX:+UseParallelGC 4 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.analysis.TestFoldingMultitermExtrasQuery Error Message: org/apache/commons/logging/LogFactory Stack Trace: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0) at org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60) at org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271) at org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127) at org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:155) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:205) at org.apache.solr.util.TestHarness.init(TestHarness.java:137) at org.apache.solr.util.TestHarness.init(TestHarness.java:147) at org.apache.solr.util.TestHarness.init(TestHarness.java:98) at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:569) at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:559) at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:379) at org.apache.solr.analysis.TestFoldingMultitermExtrasQuery.beforeTests(TestFoldingMultitermExtrasQuery.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:744) FAILED: junit.framework.TestSuite.org.apache.solr.schema.TestICUCollationField Error Message: org/apache/commons/logging/LogFactory Stack Trace: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0) at org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60) at org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271) at org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127) at
[jira] [Resolved] (SOLR-6006) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the correspond
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Rowe resolved SOLR-6006. -- Resolution: Fixed Fix Version/s: 5.0 4.9 Assignee: Steve Rowe Committed to trunk and branch_4x. Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the corresponding POMs -- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Assignee: Steve Rowe Priority: Minor Fix For: 4.9, 5.0 Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the correspon
[ https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980839#comment-13980839 ] Steve Rowe commented on SOLR-6006: -- Thanks for reporting, Steven! Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the corresponding POMs -- Key: SOLR-6006 URL: https://issues.apache.org/jira/browse/SOLR-6006 Project: Solr Issue Type: Bug Components: Build Affects Versions: 4.7.2 Reporter: Steven Scott Assignee: Steve Rowe Priority: Minor Fix For: 4.9, 5.0 Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch I'm not sure what version this first appeared in, as we just bumped from 4.5 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and without the optional flag. I checked out the source to verify that there isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able to decipher the ant build (looks like there's a pom.xml.template that generates the pom with dependencies coming from Ivy?) Anyway, this is an issue since now we have to manually exclude log4j from every project that depends on SolrJ. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1524 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1524/ Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC 4 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.analysis.TestFoldingMultitermExtrasQuery Error Message: org/apache/commons/logging/LogFactory Stack Trace: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at __randomizedtesting.SeedInfo.seed([72B2CABE3E8F95C]:0) at org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60) at org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271) at org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127) at org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:155) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:205) at org.apache.solr.util.TestHarness.init(TestHarness.java:137) at org.apache.solr.util.TestHarness.init(TestHarness.java:147) at org.apache.solr.util.TestHarness.init(TestHarness.java:98) at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:569) at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:559) at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:379) at org.apache.solr.analysis.TestFoldingMultitermExtrasQuery.beforeTests(TestFoldingMultitermExtrasQuery.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory at java.net.URLClassLoader$1.run(URLClassLoader.java:372) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:360) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 38 more FAILED: junit.framework.TestSuite.org.apache.solr.schema.TestICUCollationField Error Message: org/apache/commons/logging/LogFactory Stack Trace: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at
[jira] [Commented] (SOLR-6010) Wrong highlighting while querying by date range with wild card in the end range
[ https://issues.apache.org/jira/browse/SOLR-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980875#comment-13980875 ] Mohammad Abul Khaer commented on SOLR-6010: --- Tried that one. In that case solr return highlighting as follows {code} highlighting: { article:838: {}, article:840: {}, article:839: {}, article:824: {}, article:834: {}, article:835: {}, article:832: {}, article:833: {}, article:826: {}, article:825: {} } {code} Wrong highlighting while querying by date range with wild card in the end range --- Key: SOLR-6010 URL: https://issues.apache.org/jira/browse/SOLR-6010 Project: Solr Issue Type: Bug Components: highlighter, query parsers Affects Versions: 4.0 Environment: java version 1.7.0_45 Java(TM) SE Runtime Environment (build 1.7.0_45-b18) Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode) Linux 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Reporter: Mohammad Abul Khaer Labels: date, highlighting, range, solr Solr is returning wrong highlights when I have a date range query with wild card *in the end range*. For example my query *q* is {noformat} (porta)+activatedate:[* TO 2014-04-24T09:55:00Z]+expiredate:[2014-04-24T09:55:00Z TO *] {noformat} In the above query activatedate, expiredate are date fields. Their definition in schema file is as follows {code} field name=activatedate type=date indexed=true stored=false omitNorms=true/ field name=expiredate type=date indexed=true stored=false omitNorms=true/ {code} In the query result I am getting wrong highlighting information. Only highlighting result is show below {code} highlighting: { article:3605: { title: [ The emcreative/em emheadline/em of this emstory/em emreally/em emsays/em it emall/em ], summary: [ emEtiam/em emporta/em emsem/em emmalesuada/em emmagna/em emmollis/em emeuismod/em emaenean/em emeu/em emleo/em emquam/em. emPellentesque/em emornare/em emsem/em emlacinia/em emquam/em. ] }, article:3604: { title: [ The emcreative/em emheadline/em of this emstory/em emreally/em emsays/em it emall/em ], summary: [ emEtiam/em emporta/em emsem/em emmalesuada/em emmagna/em emmollis/em emeuismod/em emaenean/em emeu/em emleo/em emquam/em. emPellentesque/em emornare/em emsem/em emlacinia/em emquam/em.. ] } } {code} It should highlight only *story* word but it is highlighting lot other words also. What I noticed that this happens only if I have a wildcard * in the end range. If I change the above query and set a fixed date in the end range instead of * then solr return correct highlights. Modified query is shown below - {noformat} (porta)+activatedate:[* TO 2014-04-24T09:55:00Z]+expiredate:[2014-04-24T09:55:00Z TO 3014-04-24T09:55:00Z] {noformat} I guess its a bug in SOLR. If I use filter query *fq* instead of normal query *q* then highlighting result is OK for both queries. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: maxThreads in Jetty
Too avoid distributed deadlock. It would be better to add support for setting an internal port and using two thread pools - before 5x though, that would be a fairly unfriendly out of the box solution for non jetty containers. -- Mark Miller about.me/markrmiller On April 25, 2014 at 2:41:15 AM, Toke Eskildsen (t...@statsbiblioteket.dk) wrote: In the /etc/jetty.xml delivered with Solr, maxThreads is set to 10,000. I wonder why maxThreads for Jetty with Solr is so high? The default case for a Solr instance is that processing is isolated to machines controlled by the owner: Performance is dictated by local CPU, memory storage latency. There is no waiting for external services. With this in mind, I would not expect throughput to rise after a certain number of concurrent searches: When CPUs are maxed and storage is maxed, starting new Threads just increases processing time for the already running ones. A fairly beefy machine nowadays might be 24 Hyper-Threaded cores with SSDs in RAID 0 as backend. Let's say that CPU is the bottleneck here. We multiply with 2 (way too much) for the Hyper-Threading and another 2 to compensate for locks. Back of the envelope says that more than 100 concurrent searches on that machine will not increase throughput. Fewer CPUs or slower storage would only lower that number. Cache hits is a joker here as it takes very little CPU, but it would take a cache hit rate of 99% to get increased throughput from the 10,000 threads. Setting maxThread too high is bad, according to http://67-23-9-112.static.slicehost.net/doc/optimization.html The obvious problem for Solr is memory usage as some searches require a non-trivial amount of temporary heap space; notably faceting with a bitset for the hits and structures for counting (int[] or HashMap, depending on implementation). A modest index of 5M documents with 100K unique values in a facet field (used with fc or DocValues) takes 1MByte+ of memory for a single search or 10GByte+ with 10,000. The potentially huge discrepancy between the memory requirements under light load vs. heavy load seems like a trap. An unplanned burst of request might very well bring the installation down. Alternatively one could over-allocate memory to guard against OOM, but since throughput does not increase past a given amount of concurrent searches (where a given amount by my logic is far less than 10,000), this is essentially wasted: Queueing would give the same throughput with lower memory requirements. By the logic above, maxThreads of 100 or maybe 200 would be an appropriate default for Jetty with Solr. So why the 10,000? - Toke Eskildsen, State and Univeristy Library, Denmark - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: maxThreads in Jetty
On Fri, 2014-04-25 at 12:55 +0200, Mark Miller wrote: Too avoid distributed deadlock. Ah. I did not consider that. It would be better to add support for setting an internal port and using two thread pools - before 5x though, that would be a fairly unfriendly out of the box solution for non jetty containers. It is very simple in tomcat to add another connector with a given maxThreads, but of course it still involves editing a file that is otherwise not touched. I can see why that might be a problem. I do not know how difficult it is in other Web containers. Do you know if a limitation on threads is on the radar for Solr 5? Thank you, Toke Eskildsen, State and Univeristy Library, Denmark - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Lucene/Solr 4.8.0 RC2
+1 as well SUCCESS! [0:48:27.193044] On Fri, Apr 25, 2014 at 8:55 AM, Uwe Schindler u...@thetaphi.de wrote: Here is my +1: SUCCESS! [1:53:39.982427] Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Thursday, April 24, 2014 11:54 PM To: dev@lucene.apache.org Subject: [VOTE] Lucene/Solr 4.8.0 RC2 Hi, I prepared a second release candidate of Lucene and Solr 4.8.0. The artifacts can be found here: = http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0- RC2-rev1589874/ This RC contains the additional fixes for SOLR-6011, LUCENE-5626, and LUCENE-5630. Please check the artifacts and give your vote in the next 72 hrs. Uwe P.S.: Here's my smoker command line: $ JAVA_HOME=$HOME/jdk1.7.0_55 JAVA7_HOME=$HOME/jdk1.7.0_55 python3.2 -u smokeTestRelease.py ' http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-RC2- rev1589874/' 1589874 4.8.0 tmp - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6014) Nested subquery containing stop words only can invalidate whole query
[ https://issues.apache.org/jira/browse/SOLR-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Herrmann updated SOLR-6014: Description: org.apache.solr.request.StandardRequestHandler may parse the query string incorrectly when stop words like and in etc... are used within nested subqueries. Example query: {{(((_query_:\{!dismax qf=\allfields\ \}transformation) AND (_query_:\{!dismax qf=\allfields\ \}in)))}} The parsed query is: {{\+(\+(allfields:transform) ()) \+(\+() ())}} The first subquery (transformation) returns results but the second (in) does not. The term in is configured as stop word. Expected result: The query should return the same results as returned by this query: {{(((_query_:\{!dismax qf=\allfields\ \}transformation)))}}. Maybe one should just remove empty term lists? This is probably related to SOLR-261. was: org.apache.solr.request.StandardRequestHandler may parse the query string incorrectly when stop words like and in etc... are used within nested subqueries. Example query: {{(((_query_:\{!dismax qf=\allfields\ \}transformation) AND (_query_:\{!dismax qf=\allfields\ \}in)))}} The parsed query is: {{+(+(allfields:transform) ()) +(+() ())}} The first subquery (transformation) returns results but the second (in) does not. The term in is configured as stop word. Expected result: The query should return the same results as returned by this query: {{(((_query_:\{!dismax qf=\allfields\ \}transformation)))}}. Maybe one should just remove empty term lists? This is probably related to SOLR-261. Nested subquery containing stop words only can invalidate whole query - Key: SOLR-6014 URL: https://issues.apache.org/jira/browse/SOLR-6014 Project: Solr Issue Type: Bug Components: query parsers Affects Versions: 4.2.1 Reporter: Matthias Herrmann org.apache.solr.request.StandardRequestHandler may parse the query string incorrectly when stop words like and in etc... are used within nested subqueries. Example query: {{(((_query_:\{!dismax qf=\allfields\ \}transformation) AND (_query_:\{!dismax qf=\allfields\ \}in)))}} The parsed query is: {{\+(\+(allfields:transform) ()) \+(\+() ())}} The first subquery (transformation) returns results but the second (in) does not. The term in is configured as stop word. Expected result: The query should return the same results as returned by this query: {{(((_query_:\{!dismax qf=\allfields\ \}transformation)))}}. Maybe one should just remove empty term lists? This is probably related to SOLR-261. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6014) Nested subquery containing stop words only can invalidate whole query
Matthias Herrmann created SOLR-6014: --- Summary: Nested subquery containing stop words only can invalidate whole query Key: SOLR-6014 URL: https://issues.apache.org/jira/browse/SOLR-6014 Project: Solr Issue Type: Bug Components: query parsers Affects Versions: 4.2.1 Reporter: Matthias Herrmann org.apache.solr.request.StandardRequestHandler may parse the query string incorrectly when stop words like and in etc... are used within nested subqueries. Example query: {{(((_query_:\{!dismax qf=\allfields\ \}transformation) AND (_query_:\{!dismax qf=\allfields\ \}in)))}} The parsed query is: {{+(+(allfields:transform) ()) +(+() ())}} The first subquery (transformation) returns results but the second (in) does not. The term in is configured as stop word. Expected result: The query should return the same results as returned by this query: {{(((_query_:\{!dismax qf=\allfields\ \}transformation)))}}. Maybe one should just remove empty term lists? This is probably related to SOLR-261. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6015) managed synonyms don't gracefully handle lowercasing
Yonik Seeley created SOLR-6015: -- Summary: managed synonyms don't gracefully handle lowercasing Key: SOLR-6015 URL: https://issues.apache.org/jira/browse/SOLR-6015 Project: Solr Issue Type: Bug Reporter: Yonik Seeley I've been having bad luck testing new functionallity lately - the first thing I try never works ;-) /opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT http://localhost:8983/solrsis/synonyms/english; -H 'Content-type:application/json' --data-binary '{MB:[MiB,Megabyte]}' { responseHeader:{ status:500, QTime:3}, error:{ msg:Bad Request, trace:Bad Request (400) - Unsupported value null for mb; expected single value or a JSON array!\n I finally figured out that if I lowercased MB to mb, then it works as expected. Also, it looks like because ignoreCase is true in the initArgs, everything is *stored* as lower case in the managed map (losing information). Ideally case would be preserved in the synonym file (esp if one wanted to change ignoreCase either way). I imagine stopwords may have the same issue, but I haven't checked. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6015) managed synonyms don't gracefully handle lowercasing
[ https://issues.apache.org/jira/browse/SOLR-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-6015: --- Description: I've been having bad luck testing new functionallity lately - the first thing I try never works ;-) {code} /opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT http://localhost:8983/solrsis/synonyms/english; -H 'Content-type:application/json' --data-binary '{MB:[MiB,Megabyte]}' { responseHeader:{ status:500, QTime:3}, error:{ msg:Bad Request, trace:Bad Request (400) - Unsupported value null for mb; expected single value or a JSON array!\n [...] {code} I finally figured out that if I lowercased MB to mb, then it works as expected. Also, it looks like because ignoreCase is true in the initArgs, everything is *stored* as lower case in the managed map (losing information). Ideally case would be preserved in the synonym file (esp if one wanted to change ignoreCase either way). I imagine stopwords may have the same issue, but I haven't checked. was: I've been having bad luck testing new functionallity lately - the first thing I try never works ;-) /opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT http://localhost:8983/solrsis/synonyms/english; -H 'Content-type:application/json' --data-binary '{MB:[MiB,Megabyte]}' { responseHeader:{ status:500, QTime:3}, error:{ msg:Bad Request, trace:Bad Request (400) - Unsupported value null for mb; expected single value or a JSON array!\n I finally figured out that if I lowercased MB to mb, then it works as expected. Also, it looks like because ignoreCase is true in the initArgs, everything is *stored* as lower case in the managed map (losing information). Ideally case would be preserved in the synonym file (esp if one wanted to change ignoreCase either way). I imagine stopwords may have the same issue, but I haven't checked. managed synonyms don't gracefully handle lowercasing Key: SOLR-6015 URL: https://issues.apache.org/jira/browse/SOLR-6015 Project: Solr Issue Type: Bug Reporter: Yonik Seeley I've been having bad luck testing new functionallity lately - the first thing I try never works ;-) {code} /opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT http://localhost:8983/solrsis/synonyms/english; -H 'Content-type:application/json' --data-binary '{MB:[MiB,Megabyte]}' { responseHeader:{ status:500, QTime:3}, error:{ msg:Bad Request, trace:Bad Request (400) - Unsupported value null for mb; expected single value or a JSON array!\n [...] {code} I finally figured out that if I lowercased MB to mb, then it works as expected. Also, it looks like because ignoreCase is true in the initArgs, everything is *stored* as lower case in the managed map (losing information). Ideally case would be preserved in the synonym file (esp if one wanted to change ignoreCase either way). I imagine stopwords may have the same issue, but I haven't checked. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981032#comment-13981032 ] Shawn Heisey commented on SOLR-6003: Hoss is right: I had not considered dynamic fields. It should be possible to detect the *general* case of this schema might be unsuitable even with dynamic fields, as [~kduffie] described in his code example. My code example was trying to detect the *specific* case of this particular atomic update WILL lose data with the current schema. Because I haven't examined the code yet, I do not know how difficult dynamic field support will be. Because they are *probably* easy, a combination of the general case with dynamic fields and the specific case with explicit fields would be a good starting point. Test cases will be required. After that's done, hopefully dynamic fields can be fully supported in the specific case, eliminating the need for the general case. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6016) Failure indexing exampledocs with example-schemaless mode
[ https://issues.apache.org/jira/browse/SOLR-6016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-6016: Attachment: solr.log Failure indexing exampledocs with example-schemaless mode - Key: SOLR-6016 URL: https://issues.apache.org/jira/browse/SOLR-6016 Project: Solr Issue Type: Bug Components: documentation, Schema and Analysis Affects Versions: 4.7.2, 4.8 Reporter: Shalin Shekhar Mangar Attachments: solr.log Steps to reproduce: # cd example; java -Dsolr.solr.home=example-schemaless/solr -jar start.jar # cd ../exampledocs; java -jar post.jar *.xml Output from post.jar {code} Posting files to base url http://localhost:8983/solr/update using content-type application/xml.. POSTing file gb18030-example.xml POSTing file hd.xml POSTing file ipod_other.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file ipod_video.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file manufacturers.xml POSTing file mem.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file money.xml POSTing file monitor2.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file monitor.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file mp500.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file sd500.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file solr.xml POSTing file utf8-example.xml POSTing file vidcard.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update 14 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/update.. Time spent: 0:00:00.401 {code} Exceptions in Solr (I am pasting just one of them): {code} 5105 [qtp697879466-14] ERROR org.apache.solr.core.SolrCore – org.apache.solr.common.SolrException: ERROR: [doc=EN7800GTX/2DHTV/256M] Error adding field 'price'='479.95' msg=For input string: 479.95 at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167) at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77) at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:234) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) .. Caused by: java.lang.NumberFormatException: For input string: 479.95 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:441) at java.lang.Long.parseLong(Long.java:483) at org.apache.solr.schema.TrieField.createField(TrieField.java:609) at org.apache.solr.schema.TrieField.createFields(TrieField.java:660) {code} The full solr.log is attached. I understand why these errors occur but since we ship example data with Solr to demonstrate our core features, I expect that indexing exampledocs should work without errors. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail:
[jira] [Created] (SOLR-6016) Failure indexing exampledocs with example-schemaless mode
Shalin Shekhar Mangar created SOLR-6016: --- Summary: Failure indexing exampledocs with example-schemaless mode Key: SOLR-6016 URL: https://issues.apache.org/jira/browse/SOLR-6016 Project: Solr Issue Type: Bug Components: documentation, Schema and Analysis Affects Versions: 4.7.2, 4.8 Reporter: Shalin Shekhar Mangar Attachments: solr.log Steps to reproduce: # cd example; java -Dsolr.solr.home=example-schemaless/solr -jar start.jar # cd ../exampledocs; java -jar post.jar *.xml Output from post.jar {code} Posting files to base url http://localhost:8983/solr/update using content-type application/xml.. POSTing file gb18030-example.xml POSTing file hd.xml POSTing file ipod_other.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file ipod_video.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file manufacturers.xml POSTing file mem.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file money.xml POSTing file monitor2.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file monitor.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file mp500.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file sd500.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file solr.xml POSTing file utf8-example.xml POSTing file vidcard.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update 14 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/update.. Time spent: 0:00:00.401 {code} Exceptions in Solr (I am pasting just one of them): {code} 5105 [qtp697879466-14] ERROR org.apache.solr.core.SolrCore – org.apache.solr.common.SolrException: ERROR: [doc=EN7800GTX/2DHTV/256M] Error adding field 'price'='479.95' msg=For input string: 479.95 at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167) at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77) at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:234) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) .. Caused by: java.lang.NumberFormatException: For input string: 479.95 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:441) at java.lang.Long.parseLong(Long.java:483) at org.apache.solr.schema.TrieField.createField(TrieField.java:609) at org.apache.solr.schema.TrieField.createFields(TrieField.java:660) {code} The full solr.log is attached. I understand why these errors occur but since we ship example data with Solr to demonstrate our core features, I expect that indexing exampledocs should work without errors. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)
[ https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981062#comment-13981062 ] Shalin Shekhar Mangar commented on SOLR-5981: - +1 LGTM Aaron, can you tell us why you need to extend SolrWriter? I'm just curious. I don't have any objections to this change. Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected) --- Key: SOLR-5981 URL: https://issues.apache.org/jira/browse/SOLR-5981 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.0 Environment: Linux 3.13.9-200.fc20.x86_64 Solr 4.6.0 Reporter: Aaron LaBella Assignee: Shawn Heisey Priority: Minor Fix For: 4.9, 5.0 Attachments: SOLR-5981.patch Original Estimate: 1h Remaining Estimate: 1h I've been using the org.apache.solr.handler.dataimport.DataImportHandler for a bit and it's an excellent model and architecture. I'd like to extend the usage of it to plugin my own DIHWriter, but, the code doesn't allow for it. Please change ~line 227 in the DataImportHander class to be: public SolrWriter getSolrWriter instead of: private SolrWriter getSolrWriter or, at a minimum, protected, so that I can extend DataImportHandler and override this method. Thank you *sincerely* in advance for the quick turn-around on this. If the change can be made in 4.6.0 and upstream, that'd be ideal. Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981067#comment-13981067 ] Kingston Duffie edited comment on SOLR-6003 at 4/25/14 2:44 PM: Erick, I was referring to the section on Stored Values in Caveats and Limitations on this page: [https://wiki.apache.org/solr/Atomic_Updates] where it says that ALL non-copy fields must be stored. But perhaps I'm reading more into that than is actually required. If it will work to store ONLY those fields that may be updated, that would be a huge savings for us. I'm unfamiliar with the implementation. Can you confirm based on your understanding that only fields that may be updated need to be stored? If so, it would be great to update the documentation accordingly. was (Author: kduffie): @erickerickson, I was referring to the section on Stored Values in Caveats and Limitations on this page: [https://wiki.apache.org/solr/Atomic_Updates] where it says that ALL non-copy fields must be stored. But perhaps I'm reading more into that than is actually required. If it will work to store ONLY those fields that may be updated, that would be a huge savings for us. I'm unfamiliar with the implementation. Can you confirm based on your understanding that only fields that may be updated need to be stored? If so, it would be great to update the documentation accordingly. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981067#comment-13981067 ] Kingston Duffie edited comment on SOLR-6003 at 4/25/14 2:43 PM: @erickerickson, I was referring to the section on Stored Values in Caveats and Limitations on this page: [https://wiki.apache.org/solr/Atomic_Updates] where it says that ALL non-copy fields must be stored. But perhaps I'm reading more into that than is actually required. If it will work to store ONLY those fields that may be updated, that would be a huge savings for us. I'm unfamiliar with the implementation. Can you confirm based on your understanding that only fields that may be updated need to be stored? If so, it would be great to update the documentation accordingly. was (Author: kduffie): [@erickerickson], I was referring to the section on Stored Values in Caveats and Limitations on this page: [https://wiki.apache.org/solr/Atomic_Updates] where it says that ALL non-copy fields must be stored. But perhaps I'm reading more into that than is actually required. If it will work to store ONLY those fields that may be updated, that would be a huge savings for us. I'm unfamiliar with the implementation. Can you confirm based on your understanding that only fields that may be updated need to be stored? If so, it would be great to update the documentation accordingly. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981067#comment-13981067 ] Kingston Duffie commented on SOLR-6003: --- [@erickerickson], I was referring to the section on Stored Values in Caveats and Limitations on this page: [https://wiki.apache.org/solr/Atomic_Updates] where it says that ALL non-copy fields must be stored. But perhaps I'm reading more into that than is actually required. If it will work to store ONLY those fields that may be updated, that would be a huge savings for us. I'm unfamiliar with the implementation. Can you confirm based on your understanding that only fields that may be updated need to be stored? If so, it would be great to update the documentation accordingly. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Lucene/Solr 4.8.0 RC2
SUCCESS! [1:08:02.733378] Elasticsearch is happy as well... +1 simon On Fri, Apr 25, 2014 at 2:28 PM, Adrien Grand jpou...@gmail.com wrote: +1 as well SUCCESS! [0:48:27.193044] On Fri, Apr 25, 2014 at 8:55 AM, Uwe Schindler u...@thetaphi.de wrote: Here is my +1: SUCCESS! [1:53:39.982427] Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Thursday, April 24, 2014 11:54 PM To: dev@lucene.apache.org Subject: [VOTE] Lucene/Solr 4.8.0 RC2 Hi, I prepared a second release candidate of Lucene and Solr 4.8.0. The artifacts can be found here: = http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0- RC2-rev1589874/ This RC contains the additional fixes for SOLR-6011, LUCENE-5626, and LUCENE-5630. Please check the artifacts and give your vote in the next 72 hrs. Uwe P.S.: Here's my smoker command line: $ JAVA_HOME=$HOME/jdk1.7.0_55 JAVA7_HOME=$HOME/jdk1.7.0_55 python3.2 -u smokeTestRelease.py ' http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-RC2- rev1589874/' 1589874 4.8.0 tmp - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981086#comment-13981086 ] Jack Krupansky commented on SOLR-6003: -- It sounds like a separate Jira should be filed for some of these broader discussions. This specific Jira should focus on the specific issue of increment for a non-stored field, and append to a non-stored multivalued field. Clearly this case should produce an exception since it can't possibly do anything reasonable since it needs to access the previous value before applying the increment or append. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981117#comment-13981117 ] Shawn Heisey commented on SOLR-6003: [~kduffie], you are reading it correctly. When an atomic update request is made, the first thing the code does is use the uniqueKey value to query the index, then use the stored fields found in the index to populate the new SolrInputDocument object. At this point, any field that is not stored will not be present in the new SolrInputDocument. Once the new document is populated, the atomic update operations are applied. If you are using set as your action, it would not be a problem for that field to be unstored. I believe a removeAll action is planned for a future release, and that would also not be a problem with an unstored field. When the action is add or inc, there is an assumption of an existing value in the field. A new remove action is already in the code for Solr 4.9, which also has an assumption of an existing value. It would be a *MAJOR* change (probably at both the Solr and the Lucene levels) to support updating a document containing unstored fields without providing the values for those fields. It would be even harder (possibly impossible) to support doing add/inc/remove options on a field that's not stored. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981164#comment-13981164 ] Erick Erickson commented on SOLR-6003: -- Kingston: Sorry, I added to the confusion; Shawn's comment is spot on. Just to make my position on this clear, I do NOT think this kind of check if schema doesn't support updates because not enough fields are stored and throw an exception is worth the effort. While I'm an advocate of fail early and fail often, this just doesn't seem like 1 it's going to be as easy as one might think. We've already seen the effort expand with dynamic fields, what else is lurking? 2 is going to be trivial to maintain 3 is worth the risk of screwing it up and making perfectly valid, carefully thought out installations start failing because we suddenly decide to enforce rules that have never been in the contract. 4 is at all worth the complexification of the code. And assuming the stacked updates ever actually happens at the Lucene level, this will all go away anyway In particular, I'm -1,000 on anything more onerous than a warning coming out in the log files, that ideally could be turned off via configuration in the schema. I'm not convinced that just a warning would have helped Kingston Co. in this case anyway, there's a lot of info that comes out in the log file at startup, might just have been lost in the noise. FWIW JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981177#comment-13981177 ] Kingston Duffie commented on SOLR-6003: --- Shawn, your comment seems inconsistent to me. You say, you are reading it correctly, which tells me that you agree that ALL indexed fields must be stored even if you are only updating one specific field that is stored. However you go on to say that, ... it would not be a problem for that field to be unstored (as long as the action is set). Again, I'm feeling like I may just be missing something. It makes perfect sense to me that field updates to stored fields should not cause a problem even if other indexed fields are not stored. However, that's not what the documentation says. It says, (and I quote): The core functionality of atomically updating a document requires that all fields in your SchemaXml https://wiki.apache.org/solr/SchemaXml must be configured as stored=true except for fields which are copyField/ destinations -- which must be configured as stored=false. This is because the atomic updates are applied to the document represented by the existing stored field values. I assumed that this meant that after the update is applied, the full document is reindexed and if indexable fields are not all present at that time, then that information will be lost in the index. This whole bug is premised on this assumption of mine -- i.e., that it is trivial to unambiguously detect this situation and report an error. But if this assumption is incorrect, I will be delighted -- because in my case I have one small field that needs to be updated, while I would strongly prefer not to have to store other large fields that I index. (Think about changing tags associated with large documents where the body of the document will never change, but tags associated with the document will be updated later. I'd rather not store the document body.) JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)
[ https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981186#comment-13981186 ] Aaron LaBella commented on SOLR-5981: - Sure ... I did a proof of concept to use the DataImportHandler framework to import into mongodb. I think the architecture and functionality that DIH supports is fantastic (ie: evaluators, transformers, etc.), and the only import that mongodb supports (as far as I know) is a csv. So, I took advantage of the solr code base here to do everything that the DIH does, ie: connect to a DB and get data, just instead of dumping the results into the solr index, I actually create mongodb documents. Actually, my proof of concept supports two modes: insert and copy -- the former just inserts into mongodb and skips solr, the second will insert documents into both. Turns out someone else had a similar idea, but, they re-wrote half the solr dih framework: http://code.google.com/p/sql-to-nosql-importer/ My solution only requires a small extension... I'm happy to share it with the solr community if anyone else wants it. I think using mongodb as the document store and solr to index just the fields of the document you want to search on has the most potential for serious scalability. Let me know if you have any additional questions/thoughts/comments. Thanks. Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected) --- Key: SOLR-5981 URL: https://issues.apache.org/jira/browse/SOLR-5981 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.0 Environment: Linux 3.13.9-200.fc20.x86_64 Solr 4.6.0 Reporter: Aaron LaBella Assignee: Shawn Heisey Priority: Minor Fix For: 4.9, 5.0 Attachments: SOLR-5981.patch Original Estimate: 1h Remaining Estimate: 1h I've been using the org.apache.solr.handler.dataimport.DataImportHandler for a bit and it's an excellent model and architecture. I'd like to extend the usage of it to plugin my own DIHWriter, but, the code doesn't allow for it. Please change ~line 227 in the DataImportHander class to be: public SolrWriter getSolrWriter instead of: private SolrWriter getSolrWriter or, at a minimum, protected, so that I can extend DataImportHandler and override this method. Thank you *sincerely* in advance for the quick turn-around on this. If the change can be made in 4.6.0 and upstream, that'd be ideal. Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981212#comment-13981212 ] Erick Erickson commented on SOLR-6003: -- OK, we're getting lost in the vagaries of English. Here's a rule of thumb that I use. You _must_ store all fields when using atomic updates that are _not_ the destination of a copyField directive. If you don't, any data associated with a non-stored field that is _not_ the destination of a copyField directive will be lost. How's that? JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)
[ https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981214#comment-13981214 ] James Dyer commented on SOLR-5981: -- The idea with the introduction of the DIHWriter interface was that users can have DIH bring documents into other environments than Solr. But because DIH runs within Solr, it still passes the SolrWriter to DocBuilder. Really this should have been refactored not to do this (see SOLR-3671). This way you wouldn't need to be extending SolrWriter at all. Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected) --- Key: SOLR-5981 URL: https://issues.apache.org/jira/browse/SOLR-5981 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.0 Environment: Linux 3.13.9-200.fc20.x86_64 Solr 4.6.0 Reporter: Aaron LaBella Assignee: Shawn Heisey Priority: Minor Fix For: 4.9, 5.0 Attachments: SOLR-5981.patch Original Estimate: 1h Remaining Estimate: 1h I've been using the org.apache.solr.handler.dataimport.DataImportHandler for a bit and it's an excellent model and architecture. I'd like to extend the usage of it to plugin my own DIHWriter, but, the code doesn't allow for it. Please change ~line 227 in the DataImportHander class to be: public SolrWriter getSolrWriter instead of: private SolrWriter getSolrWriter or, at a minimum, protected, so that I can extend DataImportHandler and override this method. Thank you *sincerely* in advance for the quick turn-around on this. If the change can be made in 4.6.0 and upstream, that'd be ideal. Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer
Ryan Ernst created SOLR-6017: Summary: SimpleQParser uses index analyzer instead of query analyzer Key: SOLR-6017 URL: https://issues.apache.org/jira/browse/SOLR-6017 Project: Solr Issue Type: Bug Reporter: Ryan Ernst The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer(). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981230#comment-13981230 ] Kingston Duffie commented on SOLR-6003: --- Sounds unambiguous, but I want to be sure because of comments from others on this thread. Consider this hypothetical schema: field A: indexed=true, stored=true field B: indexed=true, stored=false There are no copyFields at all. With this schema, if I do an atomic update to field A on a given document, say, {a : {set : newvalue}}, I will lose information in the index about field B for that document. Correct? JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6018) Solr DataImportHandler not finding dynamic fields
Aaron LaBella created SOLR-6018: --- Summary: Solr DataImportHandler not finding dynamic fields Key: SOLR-6018 URL: https://issues.apache.org/jira/browse/SOLR-6018 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 4.7 Reporter: Aaron LaBella Fix For: 4.9 There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields around ~line 643. The logic currently says see if you can find the field from the schema, ie: SchemaField sf = schema.getFieldOrNull(key); and, if not found, go ask DIHConfiguration to find it: -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields
[ https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron LaBella updated SOLR-6018: Description: There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields around ~line 643. The logic currently says see if you can find the field from the schema, ie: SchemaField sf = schema.getFieldOrNull(key); and, if not found, go ask DIHConfiguration to find it, ie: sf = config.getSchemaField(key); The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: dynamicField name=*_Stype=string indexed=true stored=true / (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. was: There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields around ~line 643. The logic currently says see if you can find the field from the schema, ie: SchemaField sf = schema.getFieldOrNull(key); and, if not found, go ask DIHConfiguration to find it: Solr DataImportHandler not finding dynamic fields - Key: SOLR-6018 URL: https://issues.apache.org/jira/browse/SOLR-6018 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 4.7 Reporter: Aaron LaBella Fix For: 4.9 There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields around ~line 643. The logic currently says see if you can find the field from the schema, ie: SchemaField sf = schema.getFieldOrNull(key); and, if not found, go ask DIHConfiguration to find it, ie: sf = config.getSchemaField(key); The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: dynamicField name=*_Stype=string indexed=true stored=true / (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields
[ https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron LaBella updated SOLR-6018: Description: There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_Stype=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. was: There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields around ~line 643. The logic currently says see if you can find the field from the schema, ie: SchemaField sf = schema.getFieldOrNull(key); and, if not found, go ask DIHConfiguration to find it, ie: sf = config.getSchemaField(key); The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: dynamicField name=*_Stype=string indexed=true stored=true / (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. Solr DataImportHandler not finding dynamic fields - Key: SOLR-6018 URL: https://issues.apache.org/jira/browse/SOLR-6018 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 4.7 Reporter: Aaron LaBella Fix For: 4.9 Attachments: 0001-dih-config-check-for-dynamic-fields.patch There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_Stype=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields
[ https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron LaBella updated SOLR-6018: Description: There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_S type=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. was: There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_Stype=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. Solr DataImportHandler not finding dynamic fields - Key: SOLR-6018 URL: https://issues.apache.org/jira/browse/SOLR-6018 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 4.7 Reporter: Aaron LaBella Fix For: 4.9 Attachments: 0001-dih-config-check-for-dynamic-fields.patch There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_S type=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields
[ https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron LaBella updated SOLR-6018: Attachment: 0001-dih-config-check-for-dynamic-fields.patch Solr DataImportHandler not finding dynamic fields - Key: SOLR-6018 URL: https://issues.apache.org/jira/browse/SOLR-6018 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 4.7 Reporter: Aaron LaBella Fix For: 4.9 Attachments: 0001-dih-config-check-for-dynamic-fields.patch There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_Stype=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)
[ https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981251#comment-13981251 ] Aaron LaBella commented on SOLR-5981: - James, good point. But, I think the natural place to start extending things is by extending the DataImportHandler so that you can take advantage of a request handler, ie: requestHandler name=/dih class=org.apache.solr.handler.dataimport.DataImportHandler Otherwise, you have to start doing all that work yourself. Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected) --- Key: SOLR-5981 URL: https://issues.apache.org/jira/browse/SOLR-5981 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.0 Environment: Linux 3.13.9-200.fc20.x86_64 Solr 4.6.0 Reporter: Aaron LaBella Assignee: Shawn Heisey Priority: Minor Fix For: 4.9, 5.0 Attachments: SOLR-5981.patch Original Estimate: 1h Remaining Estimate: 1h I've been using the org.apache.solr.handler.dataimport.DataImportHandler for a bit and it's an excellent model and architecture. I'd like to extend the usage of it to plugin my own DIHWriter, but, the code doesn't allow for it. Please change ~line 227 in the DataImportHander class to be: public SolrWriter getSolrWriter instead of: private SolrWriter getSolrWriter or, at a minimum, protected, so that I can extend DataImportHandler and override this method. Thank you *sincerely* in advance for the quick turn-around on this. If the change can be made in 4.6.0 and upstream, that'd be ideal. Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields
[ https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron LaBella updated SOLR-6018: Description: There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match logic in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_S type=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. was: There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match login in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_S type=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. Solr DataImportHandler not finding dynamic fields - Key: SOLR-6018 URL: https://issues.apache.org/jira/browse/SOLR-6018 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 4.7 Reporter: Aaron LaBella Fix For: 4.9 Attachments: 0001-dih-config-check-for-dynamic-fields.patch There is an issue with *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643. The logic currently says see if you can find the field from the schema, ie: {code:title=DocBuilder.java|borderStyle=solid} SchemaField sf = schema.getFieldOrNull(key); {code} and, if not found, go ask DIHConfiguration to find it, ie: {code:title=DocBuilder.java|borderStyle=solid} sf = config.getSchemaField(key); {code} The latter call takes into account case-insensitivity, which is a big deal since some databases, ie: DB2, upper case all the resulting column names. In order to not modify solr-core (ie: the match logic in IndexSchema), I'm attaching a patch that makes DIHConfiguration apply the same case-insensitive logic to the DynamicFields. Without this patch, dynamic fields will not be added to the index unless you declare them like this: {code:xml} dynamicField name=*_S type=string indexed=true stored=true / {code} (note the capital S) which is in-consistent with what I believe to be solr schema conventions to have all the schema fields as lower-case. Thanks. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer
[ https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Ernst updated SOLR-6017: - Attachment: SOLR-6017.patch Patch with test. SimpleQParser uses index analyzer instead of query analyzer --- Key: SOLR-6017 URL: https://issues.apache.org/jira/browse/SOLR-6017 Project: Solr Issue Type: Bug Reporter: Ryan Ernst Attachments: SOLR-6017.patch The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer(). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981265#comment-13981265 ] Shawn Heisey commented on SOLR-6003: bq. With this schema, if I do an atomic update to field A on a given document, say, {a : {set : newvalue}}, I will lose information in the index about field B for that document. Correct? Correct. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5806) SolrCloud: UI link or script to remove node from clusterstate.json
[ https://issues.apache.org/jira/browse/SOLR-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981268#comment-13981268 ] Steven Bower commented on SOLR-5806: Would be a valuable API to have for both external cases like what is described and for the nodes themselves to decide they should go offline SolrCloud: UI link or script to remove node from clusterstate.json -- Key: SOLR-5806 URL: https://issues.apache.org/jira/browse/SOLR-5806 Project: Solr Issue Type: Improvement Affects Versions: 4.7 Reporter: Gregg Donovan Priority: Minor In cases of partial failure where a node is still connected to ZooKeeper but failing -- e.g. bad disk, bad memory, etc. -- it would be nice to have a quick UI link or command-line script to remove the node from clusterstate.json quickly. We've had partial failures where we couldn't SSH into the box but the VM was still running and connected to ZooKeeper. In these cases, we've had to power the machine down from the [ILO|http://en.wikipedia.org/wiki/HP_Integrated_Lights-Out] in order to get it out of clusterstate.json. Having something handier in such outages would be great. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981280#comment-13981280 ] Hoss Man commented on SOLR-6003: bq. With this schema, if I do an atomic update to field A on a given document, say, {a : {set : newvalue}}, I will lose information in the index about field B for that document. Correct? Correct: but that may have been your intent when designing that schema. You may have set it up that way precisely because you know that anytime you update a document you will either be explicitly supplying a replacement value for field b, or deliberately excluding a value for field b so that the current field goes away. This is what i was refering to in my earlier comment... bq. ...or the use cases i've seen in the wild where users deliberately have non-stored fields (to save space) that they might replace with new values on atomic update, or they just leave the field out of the update as a way to remove the value. (a deliberate choice of one or the other is made by their update client on every update) just because an atomic update causes information to be removed from a non-stored field, doesn't mean that removal isn't a deliberate part o the use case --- bq. This specific Jira should focus on the specific issue of increment for a non-stored field, and append to a non-stored multivalued field. Clearly this case should produce an exception since it can't possibly do anything reasonable since it needs to access the previous value before applying the increment or append. Agreed - we can't, in general, make assumptions about failing _any_ atomic update just because some fields in the schema aren't stored -- but i don't see any reason why we can't fail in some _specific_ cases of things like inc or remove directly on non-stored fields. (I'm surprised we aren't already) JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981295#comment-13981295 ] Kingston Duffie commented on SOLR-6003: --- Hoss Man, you make an excellent point that I was simply failing to think about -- the case that someone may be fine if their existing indexed content is lost for a non-stored field -- typically because they will be providing new information, if necessary, for that field and others. Thanks for helping me understand. With that in mind, I'm fine in closing this issue -- while also even more strongly looking forward to improvements in this area in future. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-2894: -- Attachment: SOLR-2894.patch I have not worked on faceting code in the past, so this is really not my area. However, here is a patch I just worked up to apply against 5x. I had to make some small changes - DateField is deprecated and there was an ndate field in the tests that could not be found. I removed it in this patch. I also fixed a few issues around licenses and formatting - this patch passes precommit except for a nocommit at the DateField change - someone should review if that change has any other ramifications. All tests pass. This code does touch some existing faceting code in a way that demands a deeper review I think, but until I have a lot more time, I'm not the man for that job. Perhaps [~hossman_luc...@fucit.org]? Implement distributed pivot faceting Key: SOLR-2894 URL: https://issues.apache.org/jira/browse/SOLR-2894 Project: Solr Issue Type: Improvement Reporter: Erik Hatcher Fix For: 4.9, 5.0 Attachments: SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, dateToObject.patch Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-6015) managed synonyms don't gracefully handle lowercasing
[ https://issues.apache.org/jira/browse/SOLR-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Potter reassigned SOLR-6015: Assignee: Timothy Potter managed synonyms don't gracefully handle lowercasing Key: SOLR-6015 URL: https://issues.apache.org/jira/browse/SOLR-6015 Project: Solr Issue Type: Bug Reporter: Yonik Seeley Assignee: Timothy Potter I've been having bad luck testing new functionallity lately - the first thing I try never works ;-) {code} /opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT http://localhost:8983/solrsis/synonyms/english; -H 'Content-type:application/json' --data-binary '{MB:[MiB,Megabyte]}' { responseHeader:{ status:500, QTime:3}, error:{ msg:Bad Request, trace:Bad Request (400) - Unsupported value null for mb; expected single value or a JSON array!\n [...] {code} I finally figured out that if I lowercased MB to mb, then it works as expected. Also, it looks like because ignoreCase is true in the initArgs, everything is *stored* as lower case in the managed map (losing information). Ideally case would be preserved in the synonym file (esp if one wanted to change ignoreCase either way). I imagine stopwords may have the same issue, but I haven't checked. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems
[ https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson closed SOLR-6003. Resolution: Not a Problem Closing as per Kingston's OK. JSON Update increment field with non-stored fields causes subtle problems - Key: SOLR-6003 URL: https://issues.apache.org/jira/browse/SOLR-6003 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.7.1 Reporter: Kingston Duffie In our application we have large multi-field documents. We occasionally need to increment one of the numeric fields or add a value to a multi-value text field. This appears to work correctly using JSON update. But later we discovered that documents were disappearing from search results and eventually found the documentation that indicates that to use field modification you must store all fields of the document. Perhaps you will argue that you need to impose this restriction -- which I would hope could be overcome because of the cost of us having to store all fields. But in any case, it would be better for others if you could return an error if someone tries to update a field on documents with non-stored fields. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer
[ https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981338#comment-13981338 ] Erick Erickson commented on SOLR-6017: -- Gaah! What do people think about refactoring getAnalyzer() to getIndexAnalyzer()? This is trappy... Actually, since it's a public method I suppose we'd have to deprecate it and just have it call a new getIndexAnalyzer(). There's only 10 or so places it's called in Solr. At least that way an IDE's autocomplete would give some clue that there even _were_ two methods SimpleQParser uses index analyzer instead of query analyzer --- Key: SOLR-6017 URL: https://issues.apache.org/jira/browse/SOLR-6017 Project: Solr Issue Type: Bug Reporter: Ryan Ernst Attachments: SOLR-6017.patch The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer(). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5145) wait w/o notify situation happening on shutdown in CoreContainer/CloserThread/SolrCores.modifyLock
[ https://issues.apache.org/jira/browse/SOLR-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981400#comment-13981400 ] Timothy Potter commented on SOLR-5145: -- I'm almost certain this is because somewhere in the code for this test, the core doesn't get closed. Look for instances of cc.getCore(name) outside of the try with resources block, i.e. should be doing: try (SolrCore core = cc.getCore(coreName)) { // code } wait w/o notify situation happening on shutdown in CoreContainer/CloserThread/SolrCores.modifyLock -- Key: SOLR-5145 URL: https://issues.apache.org/jira/browse/SOLR-5145 Project: Solr Issue Type: Bug Reporter: Hoss Man Working on SOLR-4952 i was running tests on the 4x branch and noticed TestManagedSchema.testPersistUniqueKey stalled for 1623s -- but nothing baout hte situation seems to be specific to that test. I'll attache a threaddump, but the key things i note... * TestHarness is trying to shutdown the CoreContainer * CoreContainer.shutdown(CoreContainer.java:358) is waiting on 0xf690b880 (a org.apache.solr.core.CloserThread) * CloserThread.run(CoreContainer.java:961) is waiting on 0xe068d718 (a java.lang.Object) ** that's aparently SolrCores.modifyLock * a searcherExecutor-46-thread-1 is still alive and parking to wait for 0xf66ff860 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) * no other threads seem to be doing anything -- so as far as i can tell nothing is ever going to notify on that modifyLock -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5331) nested SpanNearQuery with repeating groups does not find match
[ https://issues.apache.org/jira/browse/LUCENE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981405#comment-13981405 ] Michael Sander commented on LUCENE-5331: This bug is actually quite important to my app. I will pay whoever fixes it $500 for their efforts or donate the same to the Lucene or Apache foundations. Feel free to message me for more details. Offer expires July 2014. nested SpanNearQuery with repeating groups does not find match -- Key: LUCENE-5331 URL: https://issues.apache.org/jira/browse/LUCENE-5331 Project: Lucene - Core Issue Type: Bug Reporter: Jerry Zhou Attachments: NestedSpanNearTest.java Nested spanNear queries do not work in some cases when repeating groups are in the query. Test case is attached ... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5331) nested SpanNearQuery with repeating groups does not find match
[ https://issues.apache.org/jira/browse/LUCENE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981405#comment-13981405 ] Michael Sander edited comment on LUCENE-5331 at 4/25/14 6:23 PM: - This bug is actually quite important to my app. I will pay whoever fixes it $500 USD for their efforts or, if they prefer, donate the same to the Lucene or Apache foundations. Feel free to message me for more details. Offer expires July 2014. was (Author: speedplane): This bug is actually quite important to my app. I will pay whoever fixes it $500 for their efforts or donate the same to the Lucene or Apache foundations. Feel free to message me for more details. Offer expires July 2014. nested SpanNearQuery with repeating groups does not find match -- Key: LUCENE-5331 URL: https://issues.apache.org/jira/browse/LUCENE-5331 Project: Lucene - Core Issue Type: Bug Reporter: Jerry Zhou Attachments: NestedSpanNearTest.java Nested spanNear queries do not work in some cases when repeating groups are in the query. Test case is attached ... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5331) nested SpanNearQuery with repeating groups does not find match
[ https://issues.apache.org/jira/browse/LUCENE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981405#comment-13981405 ] Michael Sander edited comment on LUCENE-5331 at 4/25/14 6:24 PM: - This bug is actually quite important to my app. I will pay whoever fixes it $500 USD for their efforts or, if they prefer, donate the same to the Lucene or Apache foundations. Feel free to message me for more details. Offer expires July 2014. And for clarification, I am referring to the bug where a nested span returns more results than two separate spans, identified in my comment above. was (Author: speedplane): This bug is actually quite important to my app. I will pay whoever fixes it $500 USD for their efforts or, if they prefer, donate the same to the Lucene or Apache foundations. Feel free to message me for more details. Offer expires July 2014. nested SpanNearQuery with repeating groups does not find match -- Key: LUCENE-5331 URL: https://issues.apache.org/jira/browse/LUCENE-5331 Project: Lucene - Core Issue Type: Bug Reporter: Jerry Zhou Attachments: NestedSpanNearTest.java Nested spanNear queries do not work in some cases when repeating groups are in the query. Test case is attached ... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6019) Managed schema file does not show up in the Files UI
Shalin Shekhar Mangar created SOLR-6019: --- Summary: Managed schema file does not show up in the Files UI Key: SOLR-6019 URL: https://issues.apache.org/jira/browse/SOLR-6019 Project: Solr Issue Type: Bug Components: Schema and Analysis, web gui Affects Versions: 4.7.2, 4.8 Reporter: Shalin Shekhar Mangar When running with the schema-less example, I noticed that the managed-schema file does not show in the Files section of the Admin UI. This can be confusing for a user. To make sure it was not a caching issue on the browser, I closed and opened the UI again in a new tab. I also restarted Solr and still the managed-schema is not visible in the Files section. Interestingly, the schema.xml.bak does show up. A screenshot of the UI is attached. It is possible that this bug affects other managed resources as well such as synonyms but I haven't tested that yet. The schema browser works fine though. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6019) Managed schema file does not show up in the Files UI
[ https://issues.apache.org/jira/browse/SOLR-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-6019: Attachment: 6019-missing-managed-schema.png Managed schema file does not show up in the Files UI -- Key: SOLR-6019 URL: https://issues.apache.org/jira/browse/SOLR-6019 Project: Solr Issue Type: Bug Components: Schema and Analysis, web gui Affects Versions: 4.7.2, 4.8 Reporter: Shalin Shekhar Mangar Attachments: 6019-missing-managed-schema.png When running with the schema-less example, I noticed that the managed-schema file does not show in the Files section of the Admin UI. This can be confusing for a user. To make sure it was not a caching issue on the browser, I closed and opened the UI again in a new tab. I also restarted Solr and still the managed-schema is not visible in the Files section. Interestingly, the schema.xml.bak does show up. A screenshot of the UI is attached. It is possible that this bug affects other managed resources as well such as synonyms but I haven't tested that yet. The schema browser works fine though. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6016) Failure indexing exampledocs with example-schemaless mode
[ https://issues.apache.org/jira/browse/SOLR-6016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-6016: Description: Steps to reproduce: # cd example; java -Dsolr.solr.home=example-schemaless/solr -jar start.jar # cd exampledocs; java -jar post.jar *.xml Output from post.jar {code} Posting files to base url http://localhost:8983/solr/update using content-type application/xml.. POSTing file gb18030-example.xml POSTing file hd.xml POSTing file ipod_other.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file ipod_video.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file manufacturers.xml POSTing file mem.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file money.xml POSTing file monitor2.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file monitor.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file mp500.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file sd500.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file solr.xml POSTing file utf8-example.xml POSTing file vidcard.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update 14 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/update.. Time spent: 0:00:00.401 {code} Exceptions in Solr (I am pasting just one of them): {code} 5105 [qtp697879466-14] ERROR org.apache.solr.core.SolrCore – org.apache.solr.common.SolrException: ERROR: [doc=EN7800GTX/2DHTV/256M] Error adding field 'price'='479.95' msg=For input string: 479.95 at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167) at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77) at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:234) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) .. Caused by: java.lang.NumberFormatException: For input string: 479.95 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:441) at java.lang.Long.parseLong(Long.java:483) at org.apache.solr.schema.TrieField.createField(TrieField.java:609) at org.apache.solr.schema.TrieField.createFields(TrieField.java:660) {code} The full solr.log is attached. I understand why these errors occur but since we ship example data with Solr to demonstrate our core features, I expect that indexing exampledocs should work without errors. was: Steps to reproduce: # cd example; java -Dsolr.solr.home=example-schemaless/solr -jar start.jar # cd ../exampledocs; java -jar post.jar *.xml Output from post.jar {code} Posting files to base url http://localhost:8983/solr/update using content-type application/xml.. POSTing file gb18030-example.xml POSTing file hd.xml POSTing file ipod_other.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8983/solr/update POSTing file ipod_video.xml SimplePostTool: WARNING: Solr returned an error #400 Bad Request
[jira] [Created] (SOLR-6020) Auto-generate a unique key in schema-less mode if data does not have an id field
Shalin Shekhar Mangar created SOLR-6020: --- Summary: Auto-generate a unique key in schema-less mode if data does not have an id field Key: SOLR-6020 URL: https://issues.apache.org/jira/browse/SOLR-6020 Project: Solr Issue Type: Improvement Components: Schema and Analysis Reporter: Shalin Shekhar Mangar Currently it is not possible to use the schema-less example if my data does not have an id field. I was indexing data where the unique field name was url in schema-less mode. This requires one to first change unique key name in the schema and then start solr and then index docs. If one had already started solr, one'd first need to remove managed-schema, rename schema.xml.bak to schema.xml and then make the necessary changes in schema.xml. I don't think we should fail on such simple things. Here's what I propose: # We remove id and uniqueKey from the managed schema example # If there's a field named id in the document, we use that as the uniqueKey # Else we fallback on generating a UUID or a signature field via an update processor and store it as the unique key field. We can name it as id or _id # But if a uniqueKey is already present in original schema.xml then we should expect the incoming data to have that field and we should preserve the current behavior of failing loudly. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6020) Auto-generate a unique key in schema-less mode if data does not have an id field
[ https://issues.apache.org/jira/browse/SOLR-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981493#comment-13981493 ] Hoss Man commented on SOLR-6020: Wouldn't the simplest solution in this case be... * leave uniqueKey (id,string) in example-schemaless/solr/collection1/conf/managed-schema * add UUIDUpdateProcessorFactory (id) to example-schemaless/solr/collection1/conf/solconfig.xml ? UUIDUpdateProcessorFactory will already do the right thing and not generate a new ID if the document being added already has one. Auto-generate a unique key in schema-less mode if data does not have an id field -- Key: SOLR-6020 URL: https://issues.apache.org/jira/browse/SOLR-6020 Project: Solr Issue Type: Improvement Components: Schema and Analysis Reporter: Shalin Shekhar Mangar Currently it is not possible to use the schema-less example if my data does not have an id field. I was indexing data where the unique field name was url in schema-less mode. This requires one to first change unique key name in the schema and then start solr and then index docs. If one had already started solr, one'd first need to remove managed-schema, rename schema.xml.bak to schema.xml and then make the necessary changes in schema.xml. I don't think we should fail on such simple things. Here's what I propose: # We remove id and uniqueKey from the managed schema example # If there's a field named id in the document, we use that as the uniqueKey # Else we fallback on generating a UUID or a signature field via an update processor and store it as the unique key field. We can name it as id or _id # But if a uniqueKey is already present in original schema.xml then we should expect the incoming data to have that field and we should preserve the current behavior of failing loudly. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6020) Auto-generate a unique key in schema-less mode if data does not have an id field
[ https://issues.apache.org/jira/browse/SOLR-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981495#comment-13981495 ] Shalin Shekhar Mangar commented on SOLR-6020: - +1 That's even better. I wasn't aware that UUIDUpdateProcessor can do that. Auto-generate a unique key in schema-less mode if data does not have an id field -- Key: SOLR-6020 URL: https://issues.apache.org/jira/browse/SOLR-6020 Project: Solr Issue Type: Improvement Components: Schema and Analysis Reporter: Shalin Shekhar Mangar Currently it is not possible to use the schema-less example if my data does not have an id field. I was indexing data where the unique field name was url in schema-less mode. This requires one to first change unique key name in the schema and then start solr and then index docs. If one had already started solr, one'd first need to remove managed-schema, rename schema.xml.bak to schema.xml and then make the necessary changes in schema.xml. I don't think we should fail on such simple things. Here's what I propose: # We remove id and uniqueKey from the managed schema example # If there's a field named id in the document, we use that as the uniqueKey # Else we fallback on generating a UUID or a signature field via an update processor and store it as the unique key field. We can name it as id or _id # But if a uniqueKey is already present in original schema.xml then we should expect the incoming data to have that field and we should preserve the current behavior of failing loudly. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6013) Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility
[ https://issues.apache.org/jira/browse/SOLR-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981528#comment-13981528 ] Shalin Shekhar Mangar commented on SOLR-6013: - Thanks Aaron. Some comments: # DateFormatEvaluator methods such as evaluateWrapper, evaluateString, parseMathString, resolveMapper and getDateMathParser have no business being public but we can make them protected if you want. All of them should be marked as experimental API. # Evaluator.parseParams should be protected not public. # I don't like that we are creating a method such as getVariableWrapper. These things are not supposed to be pluggable and it should definitely not be public. We can mark it as protected along with a javadoc warning saying that this is experimental API. If I were writing Evaluator today, I'd just use a Callable instead. On an unrelated note, I wonder why we cache all available locales and timezones. If looking up locale/timezone is expensive then it can be done as-needed. I'll create an issue. Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility -- Key: SOLR-6013 URL: https://issues.apache.org/jira/browse/SOLR-6013 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.7 Reporter: Aaron LaBella Fix For: 4.8 Attachments: 0001-add-getters-for-datemathparser.patch, 0001-change-method-variable-visibility-and-refactor-for-extensibility.patch Original Estimate: 1h Remaining Estimate: 1h This is similar to issue 5981, the Evaluator class is declared as abstract, yet the parseParams method is package private? Surely this is an oversight, as I wouldn't expect everyone writing their own evaluators to have to deal with parsing the parameters. Similarly, I needed to refactor DateFormatEvaluator because I need to do some custom date math/parsing and it wasn't written in a way that I can extend it. Please review/apply my attached patch to the next version of Solr, ie: 4.8 or 4.9 if I must wait. Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6021) Always persist router.field in cluster state so CloudSolrServer can route documents correctly
Shalin Shekhar Mangar created SOLR-6021: --- Summary: Always persist router.field in cluster state so CloudSolrServer can route documents correctly Key: SOLR-6021 URL: https://issues.apache.org/jira/browse/SOLR-6021 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Shalin Shekhar Mangar CloudSolrServer has idField as id which is used for hashing and distributing documents. There is a setter to change it as well. IMO, we should use the correct uniqueKey automatically. I propose that we start storing router.field always in cluster state and set it to the uniqueKey field name by default. Then CloudSolrServer would not need to assume an id field by default. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6021) Always persist router.field in cluster state so CloudSolrServer can route documents correctly
[ https://issues.apache.org/jira/browse/SOLR-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981592#comment-13981592 ] Mark Miller commented on SOLR-6021: --- +1 - I had been thinking about working on this not long ago. Always persist router.field in cluster state so CloudSolrServer can route documents correctly - Key: SOLR-6021 URL: https://issues.apache.org/jira/browse/SOLR-6021 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Shalin Shekhar Mangar CloudSolrServer has idField as id which is used for hashing and distributing documents. There is a setter to change it as well. IMO, we should use the correct uniqueKey automatically. I propose that we start storing router.field always in cluster state and set it to the uniqueKey field name by default. Then CloudSolrServer would not need to assume an id field by default. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6020) Auto-generate a unique key in schema-less mode if data does not have an id field
[ https://issues.apache.org/jira/browse/SOLR-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981611#comment-13981611 ] Hoss Man commented on SOLR-6020: A related improvement that might be easy: change UUIDUpdateProcessorFactory so that if no fieldName is configured, it defaults to the uniqueKey field in the schema (if the schema has one - else error just like it does right now if you forget to configure the fieldName on the processor) Auto-generate a unique key in schema-less mode if data does not have an id field -- Key: SOLR-6020 URL: https://issues.apache.org/jira/browse/SOLR-6020 Project: Solr Issue Type: Improvement Components: Schema and Analysis Reporter: Shalin Shekhar Mangar Currently it is not possible to use the schema-less example if my data does not have an id field. I was indexing data where the unique field name was url in schema-less mode. This requires one to first change unique key name in the schema and then start solr and then index docs. If one had already started solr, one'd first need to remove managed-schema, rename schema.xml.bak to schema.xml and then make the necessary changes in schema.xml. I don't think we should fail on such simple things. Here's what I propose: # We remove id and uniqueKey from the managed schema example # If there's a field named id in the document, we use that as the uniqueKey # Else we fallback on generating a UUID or a signature field via an update processor and store it as the unique key field. We can name it as id or _id # But if a uniqueKey is already present in original schema.xml then we should expect the incoming data to have that field and we should preserve the current behavior of failing loudly. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5622) Fail tests if they print, and tests.verbose is not set
[ https://issues.apache.org/jira/browse/LUCENE-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981625#comment-13981625 ] Robert Muir commented on LUCENE-5622: - +1 to that idea. It at least prevents things from getting out of control, which is the most important. Fail tests if they print, and tests.verbose is not set -- Key: LUCENE-5622 URL: https://issues.apache.org/jira/browse/LUCENE-5622 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Assignee: Dawid Weiss Attachments: LUCENE-5622.patch, LUCENE-5622.patch, LUCENE-5622.patch, LUCENE-5622.patch Some tests print so much stuff they are now undebuggable (see LUCENE-5612). I think its bad that the testrunner hides this stuff, we used to stay on top of it. Instead, whne tests.verbose is false, we should install a printstreams (system.out/err) that fail the test instantly because they are noisy. This will ensure that our tests don't go out of control. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: VOTE: RC0 Release apache-solr-ref-guide-4.8.pdf
: 3. There are a bunch of pages with text in a much larger font size, : mixed with text in regularly-sized fonts. I noticed this on pages 71, : 85, 86, 99, 100, 118, 120, 313-317, 338, and 345. AFAICT, these all : correspond to very recent modifications. I don’t remember seeing this : problem at all in the online HTML view of this content. (I haven’t : checked yet to see if that’s still the case.) FWIW; As far as sarowe and i can figure out, these are the result of cut/pasting text from ohter documents (or web pages) in which case Confluence seems to preserve information about hte original formatting (notably the font-size) but doesn't show this formatting in hte web view or give any indication of it in the editor -- it is however used when exporting to the PDF. We think we've got them all, and i'll be putting out a new RC soon. -Hoss http://www.lucidworks.com/ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer
[ https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981631#comment-13981631 ] Ryan Ernst commented on SOLR-6017: -- bq. What do people think about refactoring getAnalyzer() to getIndexAnalyzer()? This is trappy... I completely agree. I created SOLR-6022 to address that. SimpleQParser uses index analyzer instead of query analyzer --- Key: SOLR-6017 URL: https://issues.apache.org/jira/browse/SOLR-6017 Project: Solr Issue Type: Bug Reporter: Ryan Ernst Attachments: SOLR-6017.patch The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer(). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6022) Rename getAnalyzer to getIndexAnalyzer
Ryan Ernst created SOLR-6022: Summary: Rename getAnalyzer to getIndexAnalyzer Key: SOLR-6022 URL: https://issues.apache.org/jira/browse/SOLR-6022 Project: Solr Issue Type: Improvement Reporter: Ryan Ernst We have separate index/query analyzer chains, but the access methods for the analyzers do not match up with the names. This can lead to unknowingly using the wrong analyzer chain (as it did in SOLR-6017). We should do this renaming in trunk, and deprecate the old getAnalyzer function in 4x. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer
[ https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Ernst reassigned SOLR-6017: Assignee: Ryan Ernst SimpleQParser uses index analyzer instead of query analyzer --- Key: SOLR-6017 URL: https://issues.apache.org/jira/browse/SOLR-6017 Project: Solr Issue Type: Bug Reporter: Ryan Ernst Assignee: Ryan Ernst Attachments: SOLR-6017.patch The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer(). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
VOTE: RC1 Release apache-solr-ref-guide-4.8.pdf
(Note: cross posted to general, please confine replies to dev@lucene) Please VOTE to release the following RC1 as apache-solr-ref-guide-4.8.pdf ... https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-4.8-RC1 The notes I previously mentioned regarding RC0 apply to this RC as well... 1) Due to a known bug in confluence, the PDFs it generates are much bigger then they should be. This bug has been fixed in the latest version of confluence, but cwiki.apache.rog has not yet been updated. For that reason, I have manually run a small tool against the PDF to fix the size (see SOLR-5819). The first time i tried this approach, it inadvertantly removed the Index (aka: Table of Contents, or Bookmarks depending on what PDF reader client you use). I've already fixed this, but if you notice anything else unusual about this PDF compared to previous versions please speak up so we can see if it's a result of this post-processing and try to fix it. 2) This is the first ref guide release where we've started using a special confluence macro for any lucene/solr javadoc links. The up side is that all javadoc links in this 4.8 ref guide will now correctly point to the 4.8 javadocs on lucene.apache.org -- the down side is that this means none of those links currently work, since the 4.8 code release is still ongoing and the website has not yet been updated. Because of #2, I intend to leave this ref guide vote open until the 4.8 code release is final - that way we won't officially be releasing this doc until the 4.8 javadocs are uploaded and all the links work properly. -Hoss http://www.lucidworks.com/ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5495) Recovery strategy for leader partitioned from replica case.
[ https://issues.apache.org/jira/browse/SOLR-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Potter updated SOLR-5495: - Attachment: SOLR-5495.patch Here's an updated patch that builds upon the previous one, same basic approach of leader-initiated recovery but with some added coordination between the leader and partitioned replica using a znode: /collections/collection/leader_initiated_recovery/shard/replicaCoreName (see ZkController.getLeaderInitiatedRecoveryZnodePath). The basic idea here is in addition to the leader marking the replica down, a separate znode is used to track the replica's transition from down - recovering - active. So the leader marks the replica as down (which removes it from participating in queries and update requests) and also creates this special znode. When the replica finally gets the you need to recover command from the leader, it changes the value of this znode to recovering. When recovery succeeds, the replica deletes the znode as it's no longer needed. If the leader, while trying to send the recovery command (see LeaderInitiatedRecoveryThread), sees the replica as being active but the znode wasn't ack'd, then the leader can set the state to down again. As stated before, I don't see where the replica would do this, but if it happens, we now have a better way to handle it. Bottom line is with this special znode, the replica cannot stay in the active state until it acks the leader's command to recover by transitioning the znode appropriately. The special znode is also useful if the nagging leader fails before the bad replica receives the message. The idea here is that the new leader can query ZK for any of these leader-initiated-recovery znodes for its shard and if there are any in the down state, then it can start up the nag loop for each bad replica; a znode in the down state means the replica hasn't received the recovery command yet (see: ElectionContext$ShardLeaderElectionContext.startLeaderInitiatedRecoveryOnReplicas). There is a unit test that covers the leader failing over to a new leader and resuming the nag loop on the downed replica. There's one area where I'm not 100% sure if it is correct yet ... in the shouldIBeLeader method in ShardLeaderElectionContext, I check to see if a previous leader marked this core down and if so, return false to indicate this node should not be the leader. I think this works OK for RF=3 but I'm worried about RF=2 situations where this check prevents a leader from being elected. The main idea behind this check is that if the leader forces the shard state to down, the core.getCoreDescriptor().getCloudDescriptor().getLastPublished() method can still return active so I needed this additional check on the znode. I suppose we could try to update the lastPublished state when it changes but didn't see how to go about that? (or if that was even a good idea). Another area where I'm not 100% sold on is the 10-minute max wait and then timeout loop in the LeaderInitiatedRecoveryThread. 10 mins is arbitrary but it seems like it shouldn't just run forever. One idea I had was to use JMX to raise some event / notification to allow monitoring tools to alert ops team of this issue. Curious if there's anything else in SolrCloud related to notifying of issues that need ops attention? Lastly, I did give some thought to a self-initiating recovery approach where the trying to recover loop runs on the replica itself as that is more immune to leader changes and there's already a recover retry loop in place via the RecoveryStrategy thread. As I understand it, a self-initiating approach would work something like: 1) leader receives error when trying to forward update request to replica 2) leader marks replica as down in ZK 3) replica receives state change notification (at some point), replica must iterate over all cores hosted on that node looking for cores marked as down 4) for each down core on the node found in step 3, try recovering in a loop This is all straight-forward to implement. However, the main problem with this approach is in step 4, when starting recovery, the replica updates its state to recovering in ZK immediately. When a replica is recovering the leader still tries to forward updates to it (the updates get stashed in the tlog until recovery is complete). This works in normal circumstances because the replica assumes there is no partition between it and the leader so it's OK to go into the recovering state and continue receiving updates. The problem here though is the network may still be partitioned, so the leader keeps trying to forward docs and receiving errors. From the leader's perspective, we're right back at step 1 above. Of course, it would be possible to introduce a new state that would prevent the leader from sending updates while the replica sorted itself out, but I'm hesitant to introduce a new
[jira] [Commented] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer
[ https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981698#comment-13981698 ] ASF subversion and git services commented on SOLR-6017: --- Commit 1590166 from [~rjernst] in branch 'dev/trunk' [ https://svn.apache.org/r1590166 ] SOLR-6017: Fix SimpleQParser to use query analyzer instead of index analyzer SimpleQParser uses index analyzer instead of query analyzer --- Key: SOLR-6017 URL: https://issues.apache.org/jira/browse/SOLR-6017 Project: Solr Issue Type: Bug Reporter: Ryan Ernst Assignee: Ryan Ernst Attachments: SOLR-6017.patch The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer(). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer
[ https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981716#comment-13981716 ] ASF subversion and git services commented on SOLR-6017: --- Commit 1590176 from [~rjernst] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1590176 ] SOLR-6017: Fix SimpleQParser to use query analyzer instead of index analyzer SimpleQParser uses index analyzer instead of query analyzer --- Key: SOLR-6017 URL: https://issues.apache.org/jira/browse/SOLR-6017 Project: Solr Issue Type: Bug Reporter: Ryan Ernst Assignee: Ryan Ernst Attachments: SOLR-6017.patch The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer(). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer
[ https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Ernst resolved SOLR-6017. -- Resolution: Fixed Fix Version/s: 5.0 4.9 SimpleQParser uses index analyzer instead of query analyzer --- Key: SOLR-6017 URL: https://issues.apache.org/jira/browse/SOLR-6017 Project: Solr Issue Type: Bug Reporter: Ryan Ernst Assignee: Ryan Ernst Fix For: 4.9, 5.0 Attachments: SOLR-6017.patch The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer(). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5495) Recovery strategy for leader partitioned from replica case.
[ https://issues.apache.org/jira/browse/SOLR-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981762#comment-13981762 ] Mark Miller commented on SOLR-5495: --- Awesome! Hope to read that closely this weekend. Recovery strategy for leader partitioned from replica case. --- Key: SOLR-5495 URL: https://issues.apache.org/jira/browse/SOLR-5495 Project: Solr Issue Type: Bug Reporter: Mark Miller Assignee: Timothy Potter Attachments: SOLR-5495.patch, SOLR-5495.patch We need to work out a strategy for the case of: Leader and replicas can still talk to ZooKeeper, Leader cannot talk to replica. We punted on this in initial design, but I'd like to get something in. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6019) Managed schema file does not show up in the Files UI
[ https://issues.apache.org/jira/browse/SOLR-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981767#comment-13981767 ] Stefan Matheis (steffkes) commented on SOLR-6019: - I vaguely remember another issue, which discussed that. not sure about the reasons that were mentioned - i recall it had something to do with confusing the user (or rather, try to avoid that) so in case of using managed schema, it does not have a schema.xml. Managed schema file does not show up in the Files UI -- Key: SOLR-6019 URL: https://issues.apache.org/jira/browse/SOLR-6019 Project: Solr Issue Type: Bug Components: Schema and Analysis, web gui Affects Versions: 4.7.2, 4.8 Reporter: Shalin Shekhar Mangar Attachments: 6019-missing-managed-schema.png When running with the schema-less example, I noticed that the managed-schema file does not show in the Files section of the Admin UI. This can be confusing for a user. To make sure it was not a caching issue on the browser, I closed and opened the UI again in a new tab. I also restarted Solr and still the managed-schema is not visible in the Files section. Interestingly, the schema.xml.bak does show up. A screenshot of the UI is attached. It is possible that this bug affects other managed resources as well such as synonyms but I haven't tested that yet. The schema browser works fine though. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6019) Managed schema file does not show up in the Files UI
[ https://issues.apache.org/jira/browse/SOLR-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981770#comment-13981770 ] Stefan Matheis (steffkes) commented on SOLR-6019: - found one comment in [ShowFileRequestHandler.java|http://svn.apache.org/viewvc/lucene/dev/trunk/solr/core/src/java/org/apache/solr/handler/admin/ShowFileRequestHandler.java?view=markup#l278] on line 279ff, leading towards SOLR-5455 Managed schema file does not show up in the Files UI -- Key: SOLR-6019 URL: https://issues.apache.org/jira/browse/SOLR-6019 Project: Solr Issue Type: Bug Components: Schema and Analysis, web gui Affects Versions: 4.7.2, 4.8 Reporter: Shalin Shekhar Mangar Attachments: 6019-missing-managed-schema.png When running with the schema-less example, I noticed that the managed-schema file does not show in the Files section of the Admin UI. This can be confusing for a user. To make sure it was not a caching issue on the browser, I closed and opened the UI again in a new tab. I also restarted Solr and still the managed-schema is not visible in the Files section. Interestingly, the schema.xml.bak does show up. A screenshot of the UI is attached. It is possible that this bug affects other managed resources as well such as synonyms but I haven't tested that yet. The schema browser works fine though. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20-ea-b05) - Build # 10161 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10161/ Java: 32bit/jdk1.8.0_20-ea-b05 -client -XX:+UseConcMarkSweepGC 1 tests failed. REGRESSION: org.apache.lucene.analysis.icu.TestICUNormalizer2CharFilter.testRandomStrings Error Message: startOffset 824 expected:5397 but was:5398 Stack Trace: java.lang.AssertionError: startOffset 824 expected:5397 but was:5398 at __randomizedtesting.SeedInfo.seed([FF2E1D9492AE57B7:77A71D2A31AA0082]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:181) at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:294) at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:298) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:858) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:613) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:511) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:435) at org.apache.lucene.analysis.icu.TestICUNormalizer2CharFilter.testRandomStrings(TestICUNormalizer2CharFilter.java:189) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at
[jira] [Updated] (SOLR-445) Update Handlers abort with bad documents
[ https://issues.apache.org/jira/browse/SOLR-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomás Fernández Löbbe updated SOLR-445: --- Attachment: SOLR-445-alternative.patch My simple test to use with SolrCloud fails (not 100% of the times, but very frequently). This is my understanding of the problem: It works only in the case of the update arriving to the shard leader (as it would fail while adding the doc locally), but if the update needs to be forwarded to the leader, then it will not work. If the request is forwarded to the leader it is done asynchronically and the DistributedUpdateProcessor tracks the errors internally. Finally, after all the docs where processed the “finish” method is called and the DistributedUpdateProcessor will add one of the exceptions to the response. This is a problem because “processAdd” never really fails as the TolerantUpdateProcessor is expecting. It also can’t know the total number of errors, this is counted internally in the DistributedUpdateProcessor. As a side note, this DistributedUpdateProcessor behavior makes it “tolerant”, but only in some cases? A request like this: addinvalid-doc/add addvalid-doc/add addvalid-doc/add would leave Solr in a different state depending on who is receiving the request (the shard leader or a replica/follower). Is this expected? Update Handlers abort with bad documents Key: SOLR-445 URL: https://issues.apache.org/jira/browse/SOLR-445 Project: Solr Issue Type: Improvement Components: update Affects Versions: 1.3 Reporter: Will Johnson Fix For: 4.9, 5.0 Attachments: SOLR-445-3_x.patch, SOLR-445-alternative.patch, SOLR-445-alternative.patch, SOLR-445-alternative.patch, SOLR-445-alternative.patch, SOLR-445.patch, SOLR-445.patch, SOLR-445.patch, SOLR-445.patch, SOLR-445_3x.patch, solr-445.xml Has anyone run into the problem of handling bad documents / failures mid batch. Ie: add doc field name=id1/field /doc doc field name=id2/field field name=myDateFieldI_AM_A_BAD_DATE/field /doc doc field name=id3/field /doc /add Right now solr adds the first doc and then aborts. It would seem like it should either fail the entire batch or log a message/return a code and then continue on to add doc 3. Option 1 would seem to be much harder to accomplish and possibly require more memory while Option 2 would require more information to come back from the API. I'm about to dig into this but I thought I'd ask to see if anyone had any suggestions, thoughts or comments. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: VOTE: RC1 Release apache-solr-ref-guide-4.8.pdf
+1 Steve On Fri, Apr 25, 2014 at 5:38 PM, Chris Hostetter hossman_luc...@fucit.orgwrote: (Note: cross posted to general, please confine replies to dev@lucene) Please VOTE to release the following RC1 as apache-solr-ref-guide-4.8.pdf ... https://dist.apache.org/repos/dist/dev/lucene/solr/ref- guide/apache-solr-ref-guide-4.8-RC1 The notes I previously mentioned regarding RC0 apply to this RC as well... 1) Due to a known bug in confluence, the PDFs it generates are much bigger then they should be. This bug has been fixed in the latest version of confluence, but cwiki.apache.rog has not yet been updated. For that reason, I have manually run a small tool against the PDF to fix the size (see SOLR-5819). The first time i tried this approach, it inadvertantly removed the Index (aka: Table of Contents, or Bookmarks depending on what PDF reader client you use). I've already fixed this, but if you notice anything else unusual about this PDF compared to previous versions please speak up so we can see if it's a result of this post-processing and try to fix it. 2) This is the first ref guide release where we've started using a special confluence macro for any lucene/solr javadoc links. The up side is that all javadoc links in this 4.8 ref guide will now correctly point to the 4.8 javadocs on lucene.apache.org -- the down side is that this means none of those links currently work, since the 4.8 code release is still ongoing and the website has not yet been updated. Because of #2, I intend to leave this ref guide vote open until the 4.8 code release is final - that way we won't officially be releasing this doc until the 4.8 javadocs are uploaded and all the links work properly. -Hoss http://www.lucidworks.com/
[jira] [Commented] (SOLR-5495) Recovery strategy for leader partitioned from replica case.
[ https://issues.apache.org/jira/browse/SOLR-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981852#comment-13981852 ] Otis Gospodnetic commented on SOLR-5495: bq. One idea I had was to use JMX to raise some event / notification to allow monitoring tools to alert ops team of this issue. Curious if there's anything else in SolrCloud related to notifying of issues that need ops attention? +1 for bringing this up - not that I'm aware of, but I also didn't look very closely. This may deserve a standalone JIRA! Perhaps relying on ZK for notifications would work? Recovery strategy for leader partitioned from replica case. --- Key: SOLR-5495 URL: https://issues.apache.org/jira/browse/SOLR-5495 Project: Solr Issue Type: Bug Reporter: Mark Miller Assignee: Timothy Potter Attachments: SOLR-5495.patch, SOLR-5495.patch We need to work out a strategy for the case of: Leader and replicas can still talk to ZooKeeper, Leader cannot talk to replica. We punted on this in initial design, but I'd like to get something in. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: maxThreads in Jetty
I think moving away from Jetty and going to Netty or something like that is on the radar for 5, no? Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr Elasticsearch Support * http://sematext.com/ On Fri, Apr 25, 2014 at 8:10 AM, Toke Eskildsen t...@statsbiblioteket.dkwrote: On Fri, 2014-04-25 at 12:55 +0200, Mark Miller wrote: Too avoid distributed deadlock. Ah. I did not consider that. It would be better to add support for setting an internal port and using two thread pools - before 5x though, that would be a fairly unfriendly out of the box solution for non jetty containers. It is very simple in tomcat to add another connector with a given maxThreads, but of course it still involves editing a file that is otherwise not touched. I can see why that might be a problem. I do not know how difficult it is in other Web containers. Do you know if a limitation on threads is on the radar for Solr 5? Thank you, Toke Eskildsen, State and Univeristy Library, Denmark - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6019) Managed schema file does not show up in the Files UI
[ https://issues.apache.org/jira/browse/SOLR-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981874#comment-13981874 ] Shalin Shekhar Mangar commented on SOLR-6019: - Thanks Stefan. I'd still consider this a bug. We should be able to read the file even if it cannot be edited. Also the editing feature itself is limited to trunk. Managed schema file does not show up in the Files UI -- Key: SOLR-6019 URL: https://issues.apache.org/jira/browse/SOLR-6019 Project: Solr Issue Type: Bug Components: Schema and Analysis, web gui Affects Versions: 4.7.2, 4.8 Reporter: Shalin Shekhar Mangar Attachments: 6019-missing-managed-schema.png When running with the schema-less example, I noticed that the managed-schema file does not show in the Files section of the Admin UI. This can be confusing for a user. To make sure it was not a caching issue on the browser, I closed and opened the UI again in a new tab. I also restarted Solr and still the managed-schema is not visible in the Files section. Interestingly, the schema.xml.bak does show up. A screenshot of the UI is attached. It is possible that this bug affects other managed resources as well such as synonyms but I haven't tested that yet. The schema browser works fine though. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5973) Pluggable Ranking Collectors
[ https://issues.apache.org/jira/browse/SOLR-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-5973: - Attachment: SOLR-5973.patch Added mergeStrategy test using sort criteria other then score. Pluggable Ranking Collectors Key: SOLR-5973 URL: https://issues.apache.org/jira/browse/SOLR-5973 Project: Solr Issue Type: New Feature Components: search Reporter: Joel Bernstein Assignee: Joel Bernstein Priority: Minor Fix For: 4.9 Attachments: SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch This ticket adds the ability to plugin a custom ranking collector to Solr. The proposed design is much simpler then SOLR-4465, which includes configuration support and support for pluggable analytics collectors. In this design, a CollectorFactory can be set onto the ResponseBuilder by a custom SearchComponent. The CollectorFactory is then used to inject a custom TopDocsCollector into the SolrIndexSearcher. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)
[ https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981880#comment-13981880 ] Shawn Heisey commented on SOLR-5981: [~jdyer], I'm not very familiar with DIH code. Are you saying that I shouldn't commit this patch? Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected) --- Key: SOLR-5981 URL: https://issues.apache.org/jira/browse/SOLR-5981 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.0 Environment: Linux 3.13.9-200.fc20.x86_64 Solr 4.6.0 Reporter: Aaron LaBella Assignee: Shawn Heisey Priority: Minor Fix For: 4.9, 5.0 Attachments: SOLR-5981.patch Original Estimate: 1h Remaining Estimate: 1h I've been using the org.apache.solr.handler.dataimport.DataImportHandler for a bit and it's an excellent model and architecture. I'd like to extend the usage of it to plugin my own DIHWriter, but, the code doesn't allow for it. Please change ~line 227 in the DataImportHander class to be: public SolrWriter getSolrWriter instead of: private SolrWriter getSolrWriter or, at a minimum, protected, so that I can extend DataImportHandler and override this method. Thank you *sincerely* in advance for the quick turn-around on this. If the change can be made in 4.6.0 and upstream, that'd be ideal. Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)
[ https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981887#comment-13981887 ] Shalin Shekhar Mangar commented on SOLR-5981: - bq. I did a proof of concept to use the DataImportHandler framework to import into mongodb Oh, that's awesome. I wanted to do something similar in SOLR-853 but didn't get the time to work on it. I'd like to have DIH as an API which can be used e.g. with SolrJ. Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected) --- Key: SOLR-5981 URL: https://issues.apache.org/jira/browse/SOLR-5981 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 4.0 Environment: Linux 3.13.9-200.fc20.x86_64 Solr 4.6.0 Reporter: Aaron LaBella Assignee: Shawn Heisey Priority: Minor Fix For: 4.9, 5.0 Attachments: SOLR-5981.patch Original Estimate: 1h Remaining Estimate: 1h I've been using the org.apache.solr.handler.dataimport.DataImportHandler for a bit and it's an excellent model and architecture. I'd like to extend the usage of it to plugin my own DIHWriter, but, the code doesn't allow for it. Please change ~line 227 in the DataImportHander class to be: public SolrWriter getSolrWriter instead of: private SolrWriter getSolrWriter or, at a minimum, protected, so that I can extend DataImportHandler and override this method. Thank you *sincerely* in advance for the quick turn-around on this. If the change can be made in 4.6.0 and upstream, that'd be ideal. Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org