[jira] [Updated] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that maven dependencies' scope can be specified appropriately, and the derived Maven dependencie

2014-04-25 Thread Steve Rowe (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-6006:
-

Summary: Separate test and compile scope dependencies in the Solrj ivy.xml 
file, so that maven dependencies' scope can be specified appropriately, and the 
derived Maven dependencies get filled out properly in the Solrj POM (was: 
Separate test and compile scope dependencies in ivy.xml files, so that maven 
dependencies' scope can be specified appropriately  )

 Separate test and compile scope dependencies in the Solrj ivy.xml file, so 
 that maven dependencies' scope can be specified appropriately, and the 
 derived Maven dependencies get filled out properly in the Solrj POM   
 

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Priority: Minor
 Attachments: SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM

2014-04-25 Thread Steve Rowe (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-6006:
-

Summary: Separate test and compile scope dependencies in the Solrj ivy.xml 
file, so that the derived Maven dependencies get filled out properly in the 
Solrj POM  (was: Separate test and compile scope dependencies in the Solrj 
ivy.xml file, so that maven dependencies' scope can be specified appropriately, 
and the derived Maven dependencies get filled out properly in the Solrj POM   )

 Separate test and compile scope dependencies in the Solrj ivy.xml file, so 
 that the derived Maven dependencies get filled out properly in the Solrj POM
 ---

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Priority: Minor
 Attachments: SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM

2014-04-25 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980731#comment-13980731
 ] 

ASF subversion and git services commented on SOLR-6006:
---

Commit 1589937 from [~steve_rowe] in branch 'dev/trunk'
[ https://svn.apache.org/r1589937 ]

SOLR-6006: Separate test and compile scope dependencies in the Solrj ivy.xml 
file, so that the derived Maven dependencies get filled out properly in the 
Solrj POM

 Separate test and compile scope dependencies in the Solrj ivy.xml file, so 
 that the derived Maven dependencies get filled out properly in the Solrj POM
 ---

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Priority: Minor
 Attachments: SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM

2014-04-25 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980734#comment-13980734
 ] 

ASF subversion and git services commented on SOLR-6006:
---

Commit 1589940 from [~steve_rowe] in branch 'dev/trunk'
[ https://svn.apache.org/r1589940 ]

SOLR-6006: IntelliJ config for solrj/test-lib/

 Separate test and compile scope dependencies in the Solrj ivy.xml file, so 
 that the derived Maven dependencies get filled out properly in the Solrj POM
 ---

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Priority: Minor
 Attachments: SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



maxThreads in Jetty

2014-04-25 Thread Toke Eskildsen
In the /etc/jetty.xml delivered with Solr, maxThreads is set to 10,000.
I wonder why maxThreads for Jetty with Solr is so high?


The default case for a Solr instance is that processing is isolated to
machines controlled by the owner: Performance is dictated by local CPU,
memory  storage latency. There is no waiting for external services.

With this in mind, I would not expect throughput to rise after a certain
number of concurrent searches: When CPUs are maxed and storage is maxed,
starting new Threads just increases processing time for the already
running ones.

A fairly beefy machine nowadays might be 24 Hyper-Threaded cores with
SSDs in RAID 0 as backend. Let's say that CPU is the bottleneck here. We
multiply with 2 (way too much) for the Hyper-Threading and another 2 to
compensate for locks. Back of the envelope says that more than 100
concurrent searches on that machine will not increase throughput.

Fewer CPUs or slower storage would only lower that number. Cache hits is
a joker here as it takes very little CPU, but it would take a cache hit
rate of 99% to get increased throughput from the 10,000 threads.



Setting maxThread too high is bad, according to
http://67-23-9-112.static.slicehost.net/doc/optimization.html

The obvious problem for Solr is memory usage as some searches require a
non-trivial amount of temporary heap space; notably faceting with a
bitset for the hits and structures for counting (int[] or HashMap,
depending on implementation). A modest index of 5M documents with 100K
unique values in a facet field (used with fc or DocValues) takes 1MByte+
of memory for a single search or 10GByte+ with 10,000.

The potentially huge discrepancy between the memory requirements under
light load vs. heavy load seems like a trap. An unplanned burst of
request might very well bring the installation down.

Alternatively one could over-allocate memory to guard against OOM, but
since throughput does not increase past a given amount of concurrent
searches (where a given amount by my logic is far less than 10,000),
this is essentially wasted: Queueing would give the same throughput with
lower memory requirements.


By the logic above, maxThreads of 100 or maybe 200 would be an
appropriate default for Jetty with Solr. So why the 10,000?

- Toke Eskildsen, State and Univeristy Library, Denmark



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



RE: [VOTE] Lucene/Solr 4.8.0 RC2

2014-04-25 Thread Uwe Schindler
Here is my +1:
SUCCESS! [1:53:39.982427]

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


 -Original Message-
 From: Uwe Schindler [mailto:u...@thetaphi.de]
 Sent: Thursday, April 24, 2014 11:54 PM
 To: dev@lucene.apache.org
 Subject: [VOTE] Lucene/Solr 4.8.0 RC2
 
 Hi,
 
 I prepared a second release candidate of Lucene and Solr 4.8.0. The artifacts
 can be found here:
 = http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-
 RC2-rev1589874/
 
 This RC contains the additional fixes for SOLR-6011, LUCENE-5626, and
 LUCENE-5630.
 
 Please check the artifacts and give your vote in the next 72 hrs.
 
 Uwe
 
 P.S.: Here's my smoker command line:
 $  JAVA_HOME=$HOME/jdk1.7.0_55 JAVA7_HOME=$HOME/jdk1.7.0_55
 python3.2 -u smokeTestRelease.py '
 http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-RC2-
 rev1589874/' 1589874 4.8.0 tmp
 
 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de
 
 
 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
 commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: [VOTE] Lucene/Solr 4.8.0 RC2

2014-04-25 Thread Ahmet Arslan
+1

SUCCESS! [1:46:42.182320]

Ahmet


On Friday, April 25, 2014 9:55 AM, Uwe Schindler u...@thetaphi.de wrote:
Here is my +1:
SUCCESS! [1:53:39.982427]

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


 -Original Message-
 From: Uwe Schindler [mailto:u...@thetaphi.de]
 Sent: Thursday, April 24, 2014 11:54 PM
 To: dev@lucene.apache.org
 Subject: [VOTE] Lucene/Solr 4.8.0 RC2
 
 Hi,
 
 I prepared a second release candidate of Lucene and Solr 4.8.0. The artifacts
 can be found here:
 = http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-
 RC2-rev1589874/
 
 This RC contains the additional fixes for SOLR-6011, LUCENE-5626, and
 LUCENE-5630.
 
 Please check the artifacts and give your vote in the next 72 hrs.
 
 Uwe
 
 P.S.: Here's my smoker command line:
 $  JAVA_HOME=$HOME/jdk1.7.0_55 JAVA7_HOME=$HOME/jdk1.7.0_55
 python3.2 -u smokeTestRelease.py '
 http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-RC2-
 rev1589874/' 1589874 4.8.0 tmp
 
 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de
 
 
 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
 commands, e-mail: dev-h...@lucene.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-4478) Allow cores to specify a named config set in non-SolrCloud mode

2014-04-25 Thread Alan Woodward (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980787#comment-13980787
 ] 

Alan Woodward commented on SOLR-4478:
-

Good catch!  I'll update the documentation, but really we ought not to care 
about casing.

 Allow cores to specify a named config set in non-SolrCloud mode
 ---

 Key: SOLR-4478
 URL: https://issues.apache.org/jira/browse/SOLR-4478
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2, 5.0
Reporter: Erick Erickson
Assignee: Alan Woodward
 Fix For: 4.8, 5.0

 Attachments: SOLR-4478-take2.patch, SOLR-4478-take2.patch, 
 SOLR-4478-take2.patch, SOLR-4478-take2.patch, SOLR-4478.patch, 
 SOLR-4478.patch, solr.log


 Part of moving forward to the new way, after SOLR-4196 etc... I propose an 
 additional parameter specified on the core node in solr.xml or as a 
 parameter in the discovery mode core.properties file, call it configSet, 
 where the value provided is a path to a directory, either absolute or 
 relative. Really, this is as though you copied the conf directory somewhere 
 to be used by more than one core.
 Straw-man: There will be a directory solr_home/configsets which will be the 
 default. If the configSet parameter is, say, myconf, then I'd expect a 
 directory named myconf to exist in solr_home/configsets, which would look 
 something like
 solr_home/configsets/myconf/schema.xml
   solrconfig.xml
   stopwords.txt
   velocity
   velocity/query.vm
 etc.
 If multiple cores used the same configSet, schema, solrconfig etc. would all 
 be shared (i.e. shareSchema=true would be assumed). I don't see a good 
 use-case for _not_ sharing schemas, so I don't propose to allow this to be 
 turned off. Hmmm, what if shareSchema is explicitly set to false in the 
 solr.xml or properties file? I'd guess it should be honored but maybe log a 
 warning?
 Mostly I'm putting this up for comments. I know that there are already 
 thoughts about how this all should work floating around, so before I start 
 any work on this I thought I'd at least get an idea of whether this is the 
 way people are thinking about going.
 Configset can be either a relative or absolute path, if relative it's assumed 
 to be relative to solr_home.
 Thoughts?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Early Access builds for JDK 9 b09, JDK 8u20 b10 and JDK 7U60 b15 are available on java.net

2014-04-25 Thread Rory O'Donnell Oracle, Dublin Ireland

Hi Uwe,Dawid,

Early Access builds for JDK 9 b09 https://jdk9.java.net/download/, JDK 
8u20 b10 https://jdk8.java.net/download.html and JDK 7U60 b15 
https://jdk7.java.net/download.html are available on java.net.


As we enter the later phases of development for JDK 7u60  JDK 8u20 , 
please log any show

stoppers as soon as possible.

Rgds, Rory

--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland



[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b05) - Build # 10156 - Failure!

2014-04-25 Thread Policeman Jenkins Server
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10156/
Java: 64bit/jdk1.8.0_20-ea-b05 -XX:-UseCompressedOops -XX:+UseParallelGC

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.analysis.TestFoldingMultitermExtrasQuery

Error Message:
org/apache/commons/logging/LogFactory

Stack Trace:
java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0)
at 
org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60)
at 
org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271)
at 
org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127)
at 
org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116)
at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117)
at 
org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:155)
at 
org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:205)
at org.apache.solr.util.TestHarness.init(TestHarness.java:137)
at org.apache.solr.util.TestHarness.init(TestHarness.java:147)
at org.apache.solr.util.TestHarness.init(TestHarness.java:98)
at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:569)
at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:559)
at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:379)
at 
org.apache.solr.analysis.TestFoldingMultitermExtrasQuery.beforeTests(TestFoldingMultitermExtrasQuery.java:32)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:744)


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestICUCollationField

Error Message:
org/apache/commons/logging/LogFactory

Stack Trace:
java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0)
at 
org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60)
at 
org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271)
at 
org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127)
at 
org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116)
at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117)
at 

Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b05) - Build # 10156 - Failure!

2014-04-25 Thread Steve Rowe
I’m working on a fix. - Steve

On Apr 25, 2014, at 4:25 AM, Policeman Jenkins Server jenk...@thetaphi.de 
wrote:

 Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10156/
 Java: 64bit/jdk1.8.0_20-ea-b05 -XX:-UseCompressedOops -XX:+UseParallelGC
 
 4 tests failed.
 FAILED:  
 junit.framework.TestSuite.org.apache.solr.analysis.TestFoldingMultitermExtrasQuery
 
 Error Message:
 org/apache/commons/logging/LogFactory
 
 Stack Trace:
 java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
   at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0)
   at 
 org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60)
   at 
 org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271)
   at 
 org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127)
   at 
 org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116)
   at 
 org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117)
   at 
 org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:155)
   at 
 org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49)
   at org.apache.solr.core.CoreContainer.load(CoreContainer.java:205)
   at org.apache.solr.util.TestHarness.init(TestHarness.java:137)
   at org.apache.solr.util.TestHarness.init(TestHarness.java:147)
   at org.apache.solr.util.TestHarness.init(TestHarness.java:98)
   at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:569)
   at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:559)
   at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:379)
   at 
 org.apache.solr.analysis.TestFoldingMultitermExtrasQuery.beforeTests(TestFoldingMultitermExtrasQuery.java:32)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:483)
   at 
 com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767)
   at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   at 
 com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   at 
 org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   at 
 org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   at 
 com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   at 
 com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   at 
 com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   at 
 org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   at 
 org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   at 
 org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   at 
 org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
   at java.lang.Thread.run(Thread.java:744)
 
 
 FAILED:  
 junit.framework.TestSuite.org.apache.solr.schema.TestICUCollationField
 
 Error Message:
 org/apache/commons/logging/LogFactory
 
 Stack Trace:
 java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
   at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0)
   at 
 org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60)
   at 
 org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271)
   at 
 org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127)
   at 
 org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116)
   at 
 

[jira] [Updated] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM

2014-04-25 Thread Steve Rowe (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-6006:
-

Attachment: SOLR-6006-contribs.patch

I neglected to test the Solr contribs, and they were failing after I committed 
the patch on trunk because they all have a test-scope dependency on 
jcl-over-slf4j - Uwe's Jenkins has made its unhappiness known: 
[http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10156/]

This patch fixes the build by giving each Solr contrib a {{test-lib/}} if it 
doesn't already have it, and adding jcl-over-slf4j to its test scope 
dependencies.

Committing shortly.


 Separate test and compile scope dependencies in the Solrj ivy.xml file, so 
 that the derived Maven dependencies get filled out properly in the Solrj POM
 ---

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Priority: Minor
 Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies get filled out properly in the Solrj POM

2014-04-25 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980813#comment-13980813
 ] 

ASF subversion and git services commented on SOLR-6006:
---

Commit 1589953 from [~steve_rowe] in branch 'dev/trunk'
[ https://svn.apache.org/r1589953 ]

SOLR-6006: fix Solr contrib test dependencies by adding jcl-over-slf4j and 
retrieving it into each contrib's test-lib/ directory

 Separate test and compile scope dependencies in the Solrj ivy.xml file, so 
 that the derived Maven dependencies get filled out properly in the Solrj POM
 ---

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Priority: Minor
 Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6006) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the correspondi

2014-04-25 Thread Steve Rowe (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-6006:
-

Summary: Separate test and compile scope dependencies in the Solrj and Solr 
contrib ivy.xml files, so that the derived Maven dependencies get filled out 
properly in the corresponding POMs  (was: Separate test and compile scope 
dependencies in the Solrj ivy.xml file, so that the derived Maven dependencies 
get filled out properly in the Solrj POM)

 Separate test and compile scope dependencies in the Solrj and Solr contrib 
 ivy.xml files, so that the derived Maven dependencies get filled out properly 
 in the corresponding POMs
 --

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Priority: Minor
 Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the correspon

2014-04-25 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980832#comment-13980832
 ] 

ASF subversion and git services commented on SOLR-6006:
---

Commit 1589959 from [~steve_rowe] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1589959 ]

SOLR-6006: Separate test and compile scope dependencies in the Solrj and Solr 
contrib ivy.xml files, so that the derived Maven dependencies get filled out 
properly in the corresponding POMs (merged trunk r1589937, r1589940, and 
r158953)

 Separate test and compile scope dependencies in the Solrj and Solr contrib 
 ivy.xml files, so that the derived Maven dependencies get filled out properly 
 in the corresponding POMs
 --

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Priority: Minor
 Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b05) - Build # 10156 - Failure!

2014-04-25 Thread Steve Rowe
I committed a fix. - Steve

On Apr 25, 2014, at 4:27 AM, Steve Rowe sar...@gmail.com wrote:

 I’m working on a fix. - Steve
 
 On Apr 25, 2014, at 4:25 AM, Policeman Jenkins Server jenk...@thetaphi.de 
 wrote:
 
 Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10156/
 Java: 64bit/jdk1.8.0_20-ea-b05 -XX:-UseCompressedOops -XX:+UseParallelGC
 
 4 tests failed.
 FAILED:  
 junit.framework.TestSuite.org.apache.solr.analysis.TestFoldingMultitermExtrasQuery
 
 Error Message:
 org/apache/commons/logging/LogFactory
 
 Stack Trace:
 java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
  at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0)
  at 
 org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60)
  at 
 org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271)
  at 
 org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127)
  at 
 org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116)
  at 
 org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117)
  at 
 org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:155)
  at 
 org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49)
  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:205)
  at org.apache.solr.util.TestHarness.init(TestHarness.java:137)
  at org.apache.solr.util.TestHarness.init(TestHarness.java:147)
  at org.apache.solr.util.TestHarness.init(TestHarness.java:98)
  at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:569)
  at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:559)
  at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:379)
  at 
 org.apache.solr.analysis.TestFoldingMultitermExtrasQuery.beforeTests(TestFoldingMultitermExtrasQuery.java:32)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:483)
  at 
 com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
  at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767)
  at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
  at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
 com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
  at 
 org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
  at 
 org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
  at 
 com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
  at 
 com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
  at 
 com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
  at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
 org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
  at 
 org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
  at 
 org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
  at 
 org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
  at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
  at java.lang.Thread.run(Thread.java:744)
 
 
 FAILED:  
 junit.framework.TestSuite.org.apache.solr.schema.TestICUCollationField
 
 Error Message:
 org/apache/commons/logging/LogFactory
 
 Stack Trace:
 java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
  at __randomizedtesting.SeedInfo.seed([D46B6F758DB53E8E]:0)
  at 
 org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60)
  at 
 org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271)
  at 
 org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127)
  at 
 

[jira] [Resolved] (SOLR-6006) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the correspond

2014-04-25 Thread Steve Rowe (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe resolved SOLR-6006.
--

   Resolution: Fixed
Fix Version/s: 5.0
   4.9
 Assignee: Steve Rowe

Committed to trunk and branch_4x.

 Separate test and compile scope dependencies in the Solrj and Solr contrib 
 ivy.xml files, so that the derived Maven dependencies get filled out properly 
 in the corresponding POMs
 --

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Assignee: Steve Rowe
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6006) Separate test and compile scope dependencies in the Solrj and Solr contrib ivy.xml files, so that the derived Maven dependencies get filled out properly in the correspon

2014-04-25 Thread Steve Rowe (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980839#comment-13980839
 ] 

Steve Rowe commented on SOLR-6006:
--

Thanks for reporting, Steven!

 Separate test and compile scope dependencies in the Solrj and Solr contrib 
 ivy.xml files, so that the derived Maven dependencies get filled out properly 
 in the corresponding POMs
 --

 Key: SOLR-6006
 URL: https://issues.apache.org/jira/browse/SOLR-6006
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.7.2
Reporter: Steven Scott
Assignee: Steve Rowe
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-6006-contribs.patch, SOLR-6006.patch


 I'm not sure what version this first appeared in, as we just bumped from 4.5 
 to 4.7, but log4j is specified as a dependency in the solr-solrj pom.xml, and 
 without the optional flag. I checked out the source to verify that there 
 isn't actually a dependency on log4j (doesn't seem to be), but I wasn't able 
 to decipher the ant build (looks like there's a pom.xml.template that 
 generates the pom with dependencies coming from Ivy?)
 Anyway, this is an issue since now we have to manually exclude log4j from 
 every project that depends on SolrJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1524 - Failure!

2014-04-25 Thread Policeman Jenkins Server
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1524/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.analysis.TestFoldingMultitermExtrasQuery

Error Message:
org/apache/commons/logging/LogFactory

Stack Trace:
java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at __randomizedtesting.SeedInfo.seed([72B2CABE3E8F95C]:0)
at 
org.apache.http.impl.client.CloseableHttpClient.init(CloseableHttpClient.java:60)
at 
org.apache.http.impl.client.AbstractHttpClient.init(AbstractHttpClient.java:271)
at 
org.apache.http.impl.client.DefaultHttpClient.init(DefaultHttpClient.java:127)
at 
org.apache.http.impl.client.SystemDefaultHttpClient.init(SystemDefaultHttpClient.java:116)
at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:117)
at 
org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:155)
at 
org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:205)
at org.apache.solr.util.TestHarness.init(TestHarness.java:137)
at org.apache.solr.util.TestHarness.init(TestHarness.java:147)
at org.apache.solr.util.TestHarness.init(TestHarness.java:98)
at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:569)
at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:559)
at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:379)
at 
org.apache.solr.analysis.TestFoldingMultitermExtrasQuery.beforeTests(TestFoldingMultitermExtrasQuery.java:32)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: 
org.apache.commons.logging.LogFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 38 more


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestICUCollationField

Error Message:
org/apache/commons/logging/LogFactory

Stack Trace:
java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at 

[jira] [Commented] (SOLR-6010) Wrong highlighting while querying by date range with wild card in the end range

2014-04-25 Thread Mohammad Abul Khaer (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980875#comment-13980875
 ] 

Mohammad Abul Khaer commented on SOLR-6010:
---

Tried that one. In that case solr return highlighting as follows

{code}
highlighting: {
article:838: {},
article:840: {},
article:839: {},
article:824: {},
article:834: {},
article:835: {},
article:832: {},
article:833: {},
article:826: {},
article:825: {}
  }
{code}

 Wrong highlighting while querying by date range with wild card in the end 
 range
 ---

 Key: SOLR-6010
 URL: https://issues.apache.org/jira/browse/SOLR-6010
 Project: Solr
  Issue Type: Bug
  Components: highlighter, query parsers
Affects Versions: 4.0
 Environment: java version 1.7.0_45
 Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
 Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
 Linux 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 
 x86_64 x86_64 GNU/Linux
Reporter: Mohammad Abul Khaer
  Labels: date, highlighting, range, solr

 Solr is returning wrong highlights when I have a date range query with wild 
 card *in the end range*. For example my query *q* is
 {noformat}
 (porta)+activatedate:[* TO 
 2014-04-24T09:55:00Z]+expiredate:[2014-04-24T09:55:00Z TO *]
 {noformat}
 In the above query activatedate, expiredate are date fields. Their definition 
 in schema file is as follows
 {code}
 field name=activatedate type=date indexed=true stored=false
omitNorms=true/
 field name=expiredate type=date indexed=true stored=false
omitNorms=true/
 {code}
 In the query result I am getting wrong highlighting information. Only 
 highlighting result is show below
 {code}
  highlighting: {
 article:3605: {
   title: [
 The emcreative/em emheadline/em of this emstory/em 
 emreally/em emsays/em it emall/em
   ],
   summary: [
 emEtiam/em emporta/em emsem/em emmalesuada/em 
 emmagna/em emmollis/em emeuismod/em emaenean/em emeu/em 
 emleo/em emquam/em. emPellentesque/em emornare/em 
 emsem/em emlacinia/em emquam/em.
   ]
 },
 article:3604: {
   title: [
 The emcreative/em emheadline/em of this emstory/em 
 emreally/em emsays/em it emall/em
   ],
   summary: [
 emEtiam/em emporta/em emsem/em emmalesuada/em 
 emmagna/em emmollis/em emeuismod/em emaenean/em emeu/em 
 emleo/em emquam/em. emPellentesque/em emornare/em 
 emsem/em emlacinia/em emquam/em..
   ]
 }
 }
 {code}
 It should highlight only *story* word but it is highlighting lot other words 
 also. What I noticed that this happens only if I have a wildcard * in the end 
 range. If I change the above query and set a fixed date in the end range 
 instead of * then solr return correct highlights. Modified query is shown 
 below - 
 {noformat}
 (porta)+activatedate:[* TO 
 2014-04-24T09:55:00Z]+expiredate:[2014-04-24T09:55:00Z TO 
 3014-04-24T09:55:00Z]
 {noformat}
 I guess its a bug in SOLR. If I use filter query *fq* instead of normal query 
 *q* then highlighting result is OK for both queries.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: maxThreads in Jetty

2014-04-25 Thread Mark Miller
Too avoid distributed deadlock.

It would be better to add support for setting an internal port and using two 
thread pools - before 5x though, that would be a fairly unfriendly out of the 
box solution for non jetty containers.

-- 
Mark Miller
about.me/markrmiller

On April 25, 2014 at 2:41:15 AM, Toke Eskildsen (t...@statsbiblioteket.dk) 
wrote:

In the /etc/jetty.xml delivered with Solr, maxThreads is set to 10,000.  
I wonder why maxThreads for Jetty with Solr is so high?  


The default case for a Solr instance is that processing is isolated to  
machines controlled by the owner: Performance is dictated by local CPU,  
memory  storage latency. There is no waiting for external services.  

With this in mind, I would not expect throughput to rise after a certain  
number of concurrent searches: When CPUs are maxed and storage is maxed,  
starting new Threads just increases processing time for the already  
running ones.  

A fairly beefy machine nowadays might be 24 Hyper-Threaded cores with  
SSDs in RAID 0 as backend. Let's say that CPU is the bottleneck here. We  
multiply with 2 (way too much) for the Hyper-Threading and another 2 to  
compensate for locks. Back of the envelope says that more than 100  
concurrent searches on that machine will not increase throughput.  

Fewer CPUs or slower storage would only lower that number. Cache hits is  
a joker here as it takes very little CPU, but it would take a cache hit  
rate of 99% to get increased throughput from the 10,000 threads.  



Setting maxThread too high is bad, according to  
http://67-23-9-112.static.slicehost.net/doc/optimization.html  

The obvious problem for Solr is memory usage as some searches require a  
non-trivial amount of temporary heap space; notably faceting with a  
bitset for the hits and structures for counting (int[] or HashMap,  
depending on implementation). A modest index of 5M documents with 100K  
unique values in a facet field (used with fc or DocValues) takes 1MByte+  
of memory for a single search or 10GByte+ with 10,000.  

The potentially huge discrepancy between the memory requirements under  
light load vs. heavy load seems like a trap. An unplanned burst of  
request might very well bring the installation down.  

Alternatively one could over-allocate memory to guard against OOM, but  
since throughput does not increase past a given amount of concurrent  
searches (where a given amount by my logic is far less than 10,000),  
this is essentially wasted: Queueing would give the same throughput with  
lower memory requirements.  


By the logic above, maxThreads of 100 or maybe 200 would be an  
appropriate default for Jetty with Solr. So why the 10,000?  

- Toke Eskildsen, State and Univeristy Library, Denmark  



-  
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org  
For additional commands, e-mail: dev-h...@lucene.apache.org  



Re: maxThreads in Jetty

2014-04-25 Thread Toke Eskildsen
On Fri, 2014-04-25 at 12:55 +0200, Mark Miller wrote:
 Too avoid distributed deadlock.

Ah. I did not consider that.

 It would be better to add support for setting an internal port and
 using two thread pools - before 5x though, that would be a fairly
 unfriendly out of the box solution for non jetty containers.
 
It is very simple in tomcat to add another connector with a given
maxThreads, but of course it still involves editing a file that is
otherwise not touched. I can see why that might be a problem. I do not
know how difficult it is in other Web containers.

Do you know if a limitation on threads is on the radar for Solr 5?

Thank you,
Toke Eskildsen, State and Univeristy Library, Denmark




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: [VOTE] Lucene/Solr 4.8.0 RC2

2014-04-25 Thread Adrien Grand
+1 as well
SUCCESS! [0:48:27.193044]

On Fri, Apr 25, 2014 at 8:55 AM, Uwe Schindler u...@thetaphi.de wrote:
 Here is my +1:
 SUCCESS! [1:53:39.982427]

 Uwe

 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de


 -Original Message-
 From: Uwe Schindler [mailto:u...@thetaphi.de]
 Sent: Thursday, April 24, 2014 11:54 PM
 To: dev@lucene.apache.org
 Subject: [VOTE] Lucene/Solr 4.8.0 RC2

 Hi,

 I prepared a second release candidate of Lucene and Solr 4.8.0. The artifacts
 can be found here:
 = http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-
 RC2-rev1589874/

 This RC contains the additional fixes for SOLR-6011, LUCENE-5626, and
 LUCENE-5630.

 Please check the artifacts and give your vote in the next 72 hrs.

 Uwe

 P.S.: Here's my smoker command line:
 $  JAVA_HOME=$HOME/jdk1.7.0_55 JAVA7_HOME=$HOME/jdk1.7.0_55
 python3.2 -u smokeTestRelease.py '
 http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-RC2-
 rev1589874/' 1589874 4.8.0 tmp

 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de




 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
 commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




-- 
Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6014) Nested subquery containing stop words only can invalidate whole query

2014-04-25 Thread Matthias Herrmann (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Herrmann updated SOLR-6014:


Description: 
org.apache.solr.request.StandardRequestHandler may parse the query string 
incorrectly when stop words like and in etc... are used within nested 
subqueries.

Example query:
{{(((_query_:\{!dismax qf=\allfields\ \}transformation) AND 
(_query_:\{!dismax qf=\allfields\ \}in)))}}

The parsed query is:
{{\+(\+(allfields:transform) ()) \+(\+() ())}}

The first subquery (transformation) returns results but the second (in) 
does not. The term in is configured as stop word. 

Expected result:
The query should return the same results as returned by this query: 
{{(((_query_:\{!dismax qf=\allfields\ \}transformation)))}}. Maybe one 
should just remove empty term lists?

This is probably related to SOLR-261.

  was:
org.apache.solr.request.StandardRequestHandler may parse the query string 
incorrectly when stop words like and in etc... are used within nested 
subqueries.

Example query:
{{(((_query_:\{!dismax qf=\allfields\ \}transformation) AND 
(_query_:\{!dismax qf=\allfields\ \}in)))}}

The parsed query is:
{{+(+(allfields:transform) ()) +(+() ())}}

The first subquery (transformation) returns results but the second (in) 
does not. The term in is configured as stop word. 

Expected result:
The query should return the same results as returned by this query: 
{{(((_query_:\{!dismax qf=\allfields\ \}transformation)))}}. Maybe one 
should just remove empty term lists?

This is probably related to SOLR-261.


 Nested subquery containing stop words only can invalidate whole query
 -

 Key: SOLR-6014
 URL: https://issues.apache.org/jira/browse/SOLR-6014
 Project: Solr
  Issue Type: Bug
  Components: query parsers
Affects Versions: 4.2.1
Reporter: Matthias Herrmann

 org.apache.solr.request.StandardRequestHandler may parse the query string 
 incorrectly when stop words like and in etc... are used within nested 
 subqueries.
 Example query:
 {{(((_query_:\{!dismax qf=\allfields\ \}transformation) AND 
 (_query_:\{!dismax qf=\allfields\ \}in)))}}
 The parsed query is:
 {{\+(\+(allfields:transform) ()) \+(\+() ())}}
 The first subquery (transformation) returns results but the second (in) 
 does not. The term in is configured as stop word. 
 Expected result:
 The query should return the same results as returned by this query: 
 {{(((_query_:\{!dismax qf=\allfields\ \}transformation)))}}. Maybe one 
 should just remove empty term lists?
 This is probably related to SOLR-261.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-6014) Nested subquery containing stop words only can invalidate whole query

2014-04-25 Thread Matthias Herrmann (JIRA)
Matthias Herrmann created SOLR-6014:
---

 Summary: Nested subquery containing stop words only can invalidate 
whole query
 Key: SOLR-6014
 URL: https://issues.apache.org/jira/browse/SOLR-6014
 Project: Solr
  Issue Type: Bug
  Components: query parsers
Affects Versions: 4.2.1
Reporter: Matthias Herrmann


org.apache.solr.request.StandardRequestHandler may parse the query string 
incorrectly when stop words like and in etc... are used within nested 
subqueries.

Example query:
{{(((_query_:\{!dismax qf=\allfields\ \}transformation) AND 
(_query_:\{!dismax qf=\allfields\ \}in)))}}

The parsed query is:
{{+(+(allfields:transform) ()) +(+() ())}}

The first subquery (transformation) returns results but the second (in) 
does not. The term in is configured as stop word. 

Expected result:
The query should return the same results as returned by this query: 
{{(((_query_:\{!dismax qf=\allfields\ \}transformation)))}}. Maybe one 
should just remove empty term lists?

This is probably related to SOLR-261.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-6015) managed synonyms don't gracefully handle lowercasing

2014-04-25 Thread Yonik Seeley (JIRA)
Yonik Seeley created SOLR-6015:
--

 Summary: managed synonyms don't gracefully handle lowercasing
 Key: SOLR-6015
 URL: https://issues.apache.org/jira/browse/SOLR-6015
 Project: Solr
  Issue Type: Bug
Reporter: Yonik Seeley


I've been having bad luck testing new functionallity lately - the first thing I 
try never works ;-)

/opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT 
http://localhost:8983/solrsis/synonyms/english;   -H 
'Content-type:application/json'   --data-binary '{MB:[MiB,Megabyte]}'
{
  responseHeader:{
status:500,
QTime:3},
  error:{
msg:Bad Request,
trace:Bad Request (400) - Unsupported value null for mb; expected single 
value or a JSON array!\n

I finally figured out that if I lowercased MB to mb, then it works as 
expected.
Also, it looks like because ignoreCase is true in the initArgs, everything is 
*stored* as lower case in the managed map (losing information).  Ideally case 
would be preserved in the synonym file (esp if one wanted to change ignoreCase 
either way).  I imagine stopwords may have the same issue, but I haven't 
checked.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6015) managed synonyms don't gracefully handle lowercasing

2014-04-25 Thread Yonik Seeley (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley updated SOLR-6015:
---

Description: 
I've been having bad luck testing new functionallity lately - the first thing I 
try never works ;-)

{code}
/opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT 
http://localhost:8983/solrsis/synonyms/english;   -H 
'Content-type:application/json'   --data-binary '{MB:[MiB,Megabyte]}'
{
  responseHeader:{
status:500,
QTime:3},
  error:{
msg:Bad Request,
trace:Bad Request (400) - Unsupported value null for mb; expected single 
value or a JSON array!\n
[...]
{code}

I finally figured out that if I lowercased MB to mb, then it works as 
expected.
Also, it looks like because ignoreCase is true in the initArgs, everything is 
*stored* as lower case in the managed map (losing information).  Ideally case 
would be preserved in the synonym file (esp if one wanted to change ignoreCase 
either way).  I imagine stopwords may have the same issue, but I haven't 
checked.


  was:
I've been having bad luck testing new functionallity lately - the first thing I 
try never works ;-)

/opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT 
http://localhost:8983/solrsis/synonyms/english;   -H 
'Content-type:application/json'   --data-binary '{MB:[MiB,Megabyte]}'
{
  responseHeader:{
status:500,
QTime:3},
  error:{
msg:Bad Request,
trace:Bad Request (400) - Unsupported value null for mb; expected single 
value or a JSON array!\n

I finally figured out that if I lowercased MB to mb, then it works as 
expected.
Also, it looks like because ignoreCase is true in the initArgs, everything is 
*stored* as lower case in the managed map (losing information).  Ideally case 
would be preserved in the synonym file (esp if one wanted to change ignoreCase 
either way).  I imagine stopwords may have the same issue, but I haven't 
checked.



 managed synonyms don't gracefully handle lowercasing
 

 Key: SOLR-6015
 URL: https://issues.apache.org/jira/browse/SOLR-6015
 Project: Solr
  Issue Type: Bug
Reporter: Yonik Seeley

 I've been having bad luck testing new functionallity lately - the first thing 
 I try never works ;-)
 {code}
 /opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT 
 http://localhost:8983/solrsis/synonyms/english;   -H 
 'Content-type:application/json'   --data-binary '{MB:[MiB,Megabyte]}'
 {
   responseHeader:{
 status:500,
 QTime:3},
   error:{
 msg:Bad Request,
 trace:Bad Request (400) - Unsupported value null for mb; expected 
 single value or a JSON array!\n
 [...]
 {code}
 I finally figured out that if I lowercased MB to mb, then it works as 
 expected.
 Also, it looks like because ignoreCase is true in the initArgs, everything is 
 *stored* as lower case in the managed map (losing information).  Ideally case 
 would be preserved in the synonym file (esp if one wanted to change 
 ignoreCase either way).  I imagine stopwords may have the same issue, but I 
 haven't checked.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Shawn Heisey (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981032#comment-13981032
 ] 

Shawn Heisey commented on SOLR-6003:


Hoss is right: I had not considered dynamic fields.

It should be possible to detect the *general* case of this schema might be 
unsuitable even with dynamic fields, as [~kduffie] described in his code 
example.  My code example was trying to detect the *specific* case of this 
particular atomic update WILL lose data with the current schema.  Because I 
haven't examined the code yet, I do not know how difficult dynamic field 
support will be.

Because they are *probably* easy, a combination of the general case with 
dynamic fields and the specific case with explicit fields would be a good 
starting point.  Test cases will be required.  After that's done, hopefully 
dynamic fields can be fully supported in the specific case, eliminating the 
need for the general case.

 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6016) Failure indexing exampledocs with example-schemaless mode

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-6016:


Attachment: solr.log

 Failure indexing exampledocs with example-schemaless mode
 -

 Key: SOLR-6016
 URL: https://issues.apache.org/jira/browse/SOLR-6016
 Project: Solr
  Issue Type: Bug
  Components: documentation, Schema and Analysis
Affects Versions: 4.7.2, 4.8
Reporter: Shalin Shekhar Mangar
 Attachments: solr.log


 Steps to reproduce:
 # cd example; java -Dsolr.solr.home=example-schemaless/solr -jar start.jar
 # cd ../exampledocs; java -jar post.jar *.xml
 Output from post.jar
 {code}
 Posting files to base url http://localhost:8983/solr/update using 
 content-type application/xml..
 POSTing file gb18030-example.xml
 POSTing file hd.xml
 POSTing file ipod_other.xml
 SimplePostTool: WARNING: Solr returned an error #400 Bad Request
 SimplePostTool: WARNING: IOException while reading response: 
 java.io.IOException: Server returned HTTP response code: 400 for URL: 
 http://localhost:8983/solr/update
 POSTing file ipod_video.xml
 SimplePostTool: WARNING: Solr returned an error #400 Bad Request
 SimplePostTool: WARNING: IOException while reading response: 
 java.io.IOException: Server returned HTTP response code: 400 for URL: 
 http://localhost:8983/solr/update
 POSTing file manufacturers.xml
 POSTing file mem.xml
 SimplePostTool: WARNING: Solr returned an error #400 Bad Request
 SimplePostTool: WARNING: IOException while reading response: 
 java.io.IOException: Server returned HTTP response code: 400 for URL: 
 http://localhost:8983/solr/update
 POSTing file money.xml
 POSTing file monitor2.xml
 SimplePostTool: WARNING: Solr returned an error #400 Bad Request
 SimplePostTool: WARNING: IOException while reading response: 
 java.io.IOException: Server returned HTTP response code: 400 for URL: 
 http://localhost:8983/solr/update
 POSTing file monitor.xml
 SimplePostTool: WARNING: Solr returned an error #400 Bad Request
 SimplePostTool: WARNING: IOException while reading response: 
 java.io.IOException: Server returned HTTP response code: 400 for URL: 
 http://localhost:8983/solr/update
 POSTing file mp500.xml
 SimplePostTool: WARNING: Solr returned an error #400 Bad Request
 SimplePostTool: WARNING: IOException while reading response: 
 java.io.IOException: Server returned HTTP response code: 400 for URL: 
 http://localhost:8983/solr/update
 POSTing file sd500.xml
 SimplePostTool: WARNING: Solr returned an error #400 Bad Request
 SimplePostTool: WARNING: IOException while reading response: 
 java.io.IOException: Server returned HTTP response code: 400 for URL: 
 http://localhost:8983/solr/update
 POSTing file solr.xml
 POSTing file utf8-example.xml
 POSTing file vidcard.xml
 SimplePostTool: WARNING: Solr returned an error #400 Bad Request
 SimplePostTool: WARNING: IOException while reading response: 
 java.io.IOException: Server returned HTTP response code: 400 for URL: 
 http://localhost:8983/solr/update
 14 files indexed.
 COMMITting Solr index changes to http://localhost:8983/solr/update..
 Time spent: 0:00:00.401
 {code}
 Exceptions in Solr (I am pasting just one of them):
 {code}
 5105 [qtp697879466-14] ERROR org.apache.solr.core.SolrCore  – 
 org.apache.solr.common.SolrException: ERROR: [doc=EN7800GTX/2DHTV/256M] Error 
 adding field 'price'='479.95' msg=For input string: 479.95
   at 
 org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167)
   at 
 org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77)
   at 
 org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:234)
   at 
 org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160)
   at 
 org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
   at 
 org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
 ..
 Caused by: java.lang.NumberFormatException: For input string: 479.95
   at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
   at java.lang.Long.parseLong(Long.java:441)
   at java.lang.Long.parseLong(Long.java:483)
   at org.apache.solr.schema.TrieField.createField(TrieField.java:609)
   at org.apache.solr.schema.TrieField.createFields(TrieField.java:660)
 {code}
 The full solr.log is attached.
 I understand why these errors occur but since we ship example data with Solr 
 to demonstrate our core features, I expect that indexing exampledocs should 
 work without errors.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: 

[jira] [Created] (SOLR-6016) Failure indexing exampledocs with example-schemaless mode

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)
Shalin Shekhar Mangar created SOLR-6016:
---

 Summary: Failure indexing exampledocs with example-schemaless mode
 Key: SOLR-6016
 URL: https://issues.apache.org/jira/browse/SOLR-6016
 Project: Solr
  Issue Type: Bug
  Components: documentation, Schema and Analysis
Affects Versions: 4.7.2, 4.8
Reporter: Shalin Shekhar Mangar
 Attachments: solr.log

Steps to reproduce:
# cd example; java -Dsolr.solr.home=example-schemaless/solr -jar start.jar
# cd ../exampledocs; java -jar post.jar *.xml

Output from post.jar
{code}
Posting files to base url http://localhost:8983/solr/update using content-type 
application/xml..
POSTing file gb18030-example.xml
POSTing file hd.xml
POSTing file ipod_other.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file ipod_video.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file manufacturers.xml
POSTing file mem.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file money.xml
POSTing file monitor2.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file monitor.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file mp500.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file sd500.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file solr.xml
POSTing file utf8-example.xml
POSTing file vidcard.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
14 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/update..
Time spent: 0:00:00.401
{code}

Exceptions in Solr (I am pasting just one of them):
{code}
5105 [qtp697879466-14] ERROR org.apache.solr.core.SolrCore  – 
org.apache.solr.common.SolrException: ERROR: [doc=EN7800GTX/2DHTV/256M] Error 
adding field 'price'='479.95' msg=For input string: 479.95
at 
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167)
at 
org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:234)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160)
at 
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
at 
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
..
Caused by: java.lang.NumberFormatException: For input string: 479.95
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:441)
at java.lang.Long.parseLong(Long.java:483)
at org.apache.solr.schema.TrieField.createField(TrieField.java:609)
at org.apache.solr.schema.TrieField.createFields(TrieField.java:660)
{code}

The full solr.log is attached.

I understand why these errors occur but since we ship example data with Solr to 
demonstrate our core features, I expect that indexing exampledocs should work 
without errors.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981062#comment-13981062
 ] 

Shalin Shekhar Mangar commented on SOLR-5981:
-

+1 LGTM

Aaron, can you tell us why you need to extend SolrWriter? I'm just curious. I 
don't have any objections to this change.

 Please change method visibility of getSolrWriter in DataImportHandler to 
 public (or at least protected)
 ---

 Key: SOLR-5981
 URL: https://issues.apache.org/jira/browse/SOLR-5981
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.0
 Environment: Linux 3.13.9-200.fc20.x86_64
 Solr 4.6.0
Reporter: Aaron LaBella
Assignee: Shawn Heisey
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-5981.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 I've been using the org.apache.solr.handler.dataimport.DataImportHandler for 
 a bit and it's an excellent model and architecture.  I'd like to extend the 
 usage of it to plugin my own DIHWriter, but, the code doesn't allow for it.  
 Please change ~line 227 in the DataImportHander class to be:
 public SolrWriter getSolrWriter
 instead of:
 private SolrWriter getSolrWriter
 or, at a minimum, protected, so that I can extend DataImportHandler and 
 override this method.
 Thank you *sincerely* in advance for the quick turn-around on this.  If the 
 change can be made in 4.6.0 and upstream, that'd be ideal.
 Thanks!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Kingston Duffie (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981067#comment-13981067
 ] 

Kingston Duffie edited comment on SOLR-6003 at 4/25/14 2:44 PM:


Erick,  I was referring to the section on Stored Values in Caveats and 
Limitations on this page:  [https://wiki.apache.org/solr/Atomic_Updates]  where 
it says that ALL non-copy fields must be stored.  But perhaps I'm reading more 
into that than is actually required.  If it will work to store ONLY those 
fields that may be updated, that would be a huge savings for us.  I'm 
unfamiliar with the implementation.  Can you confirm based on your 
understanding that only fields that may be updated need to be stored?  If so, 
it would be great to update the documentation accordingly.


was (Author: kduffie):
@erickerickson,  I was referring to the section on Stored Values in Caveats and 
Limitations on this page:  [https://wiki.apache.org/solr/Atomic_Updates]  where 
it says that ALL non-copy fields must be stored.  But perhaps I'm reading more 
into that than is actually required.  If it will work to store ONLY those 
fields that may be updated, that would be a huge savings for us.  I'm 
unfamiliar with the implementation.  Can you confirm based on your 
understanding that only fields that may be updated need to be stored?  If so, 
it would be great to update the documentation accordingly.

 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Kingston Duffie (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981067#comment-13981067
 ] 

Kingston Duffie edited comment on SOLR-6003 at 4/25/14 2:43 PM:


@erickerickson,  I was referring to the section on Stored Values in Caveats and 
Limitations on this page:  [https://wiki.apache.org/solr/Atomic_Updates]  where 
it says that ALL non-copy fields must be stored.  But perhaps I'm reading more 
into that than is actually required.  If it will work to store ONLY those 
fields that may be updated, that would be a huge savings for us.  I'm 
unfamiliar with the implementation.  Can you confirm based on your 
understanding that only fields that may be updated need to be stored?  If so, 
it would be great to update the documentation accordingly.


was (Author: kduffie):
[@erickerickson],  I was referring to the section on Stored Values in Caveats 
and Limitations on this page:  [https://wiki.apache.org/solr/Atomic_Updates]  
where it says that ALL non-copy fields must be stored.  But perhaps I'm reading 
more into that than is actually required.  If it will work to store ONLY those 
fields that may be updated, that would be a huge savings for us.  I'm 
unfamiliar with the implementation.  Can you confirm based on your 
understanding that only fields that may be updated need to be stored?  If so, 
it would be great to update the documentation accordingly.

 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Kingston Duffie (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981067#comment-13981067
 ] 

Kingston Duffie commented on SOLR-6003:
---

[@erickerickson],  I was referring to the section on Stored Values in Caveats 
and Limitations on this page:  [https://wiki.apache.org/solr/Atomic_Updates]  
where it says that ALL non-copy fields must be stored.  But perhaps I'm reading 
more into that than is actually required.  If it will work to store ONLY those 
fields that may be updated, that would be a huge savings for us.  I'm 
unfamiliar with the implementation.  Can you confirm based on your 
understanding that only fields that may be updated need to be stored?  If so, 
it would be great to update the documentation accordingly.

 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: [VOTE] Lucene/Solr 4.8.0 RC2

2014-04-25 Thread Simon Willnauer
SUCCESS! [1:08:02.733378]

Elasticsearch is happy as well...

+1

simon

On Fri, Apr 25, 2014 at 2:28 PM, Adrien Grand jpou...@gmail.com wrote:
 +1 as well
 SUCCESS! [0:48:27.193044]

 On Fri, Apr 25, 2014 at 8:55 AM, Uwe Schindler u...@thetaphi.de wrote:
 Here is my +1:
 SUCCESS! [1:53:39.982427]

 Uwe

 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de


 -Original Message-
 From: Uwe Schindler [mailto:u...@thetaphi.de]
 Sent: Thursday, April 24, 2014 11:54 PM
 To: dev@lucene.apache.org
 Subject: [VOTE] Lucene/Solr 4.8.0 RC2

 Hi,

 I prepared a second release candidate of Lucene and Solr 4.8.0. The 
 artifacts
 can be found here:
 = http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-
 RC2-rev1589874/

 This RC contains the additional fixes for SOLR-6011, LUCENE-5626, and
 LUCENE-5630.

 Please check the artifacts and give your vote in the next 72 hrs.

 Uwe

 P.S.: Here's my smoker command line:
 $  JAVA_HOME=$HOME/jdk1.7.0_55 JAVA7_HOME=$HOME/jdk1.7.0_55
 python3.2 -u smokeTestRelease.py '
 http://people.apache.org/~uschindler/staging_area/lucene-solr-4.8.0-RC2-
 rev1589874/' 1589874 4.8.0 tmp

 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de




 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
 commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Jack Krupansky (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981086#comment-13981086
 ] 

Jack Krupansky commented on SOLR-6003:
--

It sounds like a separate Jira should be filed for some of these broader 
discussions.

This specific Jira should focus on the specific issue of increment for a 
non-stored field, and append to a non-stored multivalued field. Clearly this 
case should produce an exception since it can't possibly do anything reasonable 
since it needs to access the previous value before applying the increment or 
append.


 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Shawn Heisey (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981117#comment-13981117
 ] 

Shawn Heisey commented on SOLR-6003:


[~kduffie], you are reading it correctly.

When an atomic update request is made, the first thing the code does is use the 
uniqueKey value to query the index, then use the stored fields found in the 
index to populate the new SolrInputDocument object.  At this point, any field 
that is not stored will not be present in the new SolrInputDocument.  Once the 
new document is populated, the atomic update operations are applied.

If you are using set as your action, it would not be a problem for that field 
to be unstored.  I believe a removeAll action is planned for a future 
release, and that would also not be a problem with an unstored field.

When the action is add or inc, there is an assumption of an existing value 
in the field.  A new remove action is already in the code for Solr 4.9, which 
also has an assumption of an existing value.

It would be a *MAJOR* change (probably at both the Solr and the Lucene levels) 
to support updating a document containing unstored fields without providing the 
values for those fields.  It would be even harder (possibly impossible) to 
support doing add/inc/remove options on a field that's not stored.


 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Erick Erickson (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981164#comment-13981164
 ] 

Erick Erickson commented on SOLR-6003:
--

Kingston:

Sorry, I added to the confusion; Shawn's comment is spot on.

Just to make my position on this clear, I do NOT think this kind of check if 
schema doesn't support updates because not enough fields are stored and throw 
an exception is worth the effort. While I'm an advocate of fail early and 
fail often, this just doesn't seem like 
1 it's going to be as easy as one might think. We've already seen the effort 
expand with dynamic fields, what else is lurking?
2 is going to be trivial to maintain
3 is worth the risk of screwing it up and making perfectly valid, carefully 
thought out installations start failing because we suddenly decide to enforce 
rules that have never been in the contract.
4 is at all worth the complexification of the code.

And assuming the stacked updates ever actually happens at the Lucene level, 
this will all go away anyway

In particular, I'm -1,000 on anything more onerous than a warning coming out in 
the log files, that ideally could be turned off via configuration in the 
schema. I'm not convinced that just a warning would have helped Kingston  Co. 
in this case anyway, there's a lot of info that comes out in the log file at 
startup, might just have been lost in the noise.

FWIW

 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Kingston Duffie (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981177#comment-13981177
 ] 

Kingston Duffie commented on SOLR-6003:
---

Shawn, your comment seems inconsistent to me.  You say, you are reading it
correctly, which tells me that you agree that ALL indexed fields must be
stored even if you are only updating one specific field that is stored.
 However you go on to say that, ... it would not be a problem for that
field to be unstored (as long as the action is set).

Again, I'm feeling like I may just be missing something.  It makes perfect
sense to me that field updates to stored fields should not cause a problem
even if other indexed fields are not stored.  However, that's not what the
documentation says.  It says, (and I quote):

The core functionality of atomically updating a document requires that all
fields in your SchemaXml https://wiki.apache.org/solr/SchemaXml must be
configured as stored=true except for fields which are copyField/
destinations
-- which must be configured as stored=false. This is because the atomic
updates are applied to the document represented by the existing stored
field values.

I assumed that this meant that after the update is applied, the full
document is reindexed and if indexable fields are not all present at that
time, then that information will be lost in the index.

This whole bug is premised on this assumption of mine -- i.e., that it is
trivial to unambiguously detect this situation and report an error.  But if
this assumption is incorrect, I will be delighted -- because in my case I
have one small field that needs to be updated, while I would strongly
prefer not to have to store other large fields that I index.  (Think about
changing tags associated with large documents where the body of the
document will never change, but tags associated with the document will be
updated later.  I'd rather not store the document body.)





 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)

2014-04-25 Thread Aaron LaBella (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981186#comment-13981186
 ] 

Aaron LaBella commented on SOLR-5981:
-

Sure ... I did a proof of concept to use the DataImportHandler framework to 
import into mongodb.  I think the architecture and functionality that DIH 
supports is fantastic (ie: evaluators, transformers, etc.), and the only 
import that mongodb supports (as far as I know) is a csv.

So, I took advantage of the solr code base here to do everything that the DIH 
does, ie: connect to a DB and get data, just instead of dumping the results 
into the solr index, I actually create mongodb documents.  Actually, my proof 
of concept supports two modes: insert and copy -- the former just inserts into 
mongodb and skips solr, the second will insert documents into both.

Turns out someone else had a similar idea, but, they re-wrote half the solr dih 
framework:
http://code.google.com/p/sql-to-nosql-importer/

My solution only requires a small extension... I'm happy to share it with the 
solr community if anyone else wants it.  I think using mongodb as the document 
store and solr to index just the fields of the document you want to search on 
has the most potential for serious scalability.

Let me know if you have any additional questions/thoughts/comments.

Thanks.

 Please change method visibility of getSolrWriter in DataImportHandler to 
 public (or at least protected)
 ---

 Key: SOLR-5981
 URL: https://issues.apache.org/jira/browse/SOLR-5981
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.0
 Environment: Linux 3.13.9-200.fc20.x86_64
 Solr 4.6.0
Reporter: Aaron LaBella
Assignee: Shawn Heisey
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-5981.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 I've been using the org.apache.solr.handler.dataimport.DataImportHandler for 
 a bit and it's an excellent model and architecture.  I'd like to extend the 
 usage of it to plugin my own DIHWriter, but, the code doesn't allow for it.  
 Please change ~line 227 in the DataImportHander class to be:
 public SolrWriter getSolrWriter
 instead of:
 private SolrWriter getSolrWriter
 or, at a minimum, protected, so that I can extend DataImportHandler and 
 override this method.
 Thank you *sincerely* in advance for the quick turn-around on this.  If the 
 change can be made in 4.6.0 and upstream, that'd be ideal.
 Thanks!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Erick Erickson (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981212#comment-13981212
 ] 

Erick Erickson commented on SOLR-6003:
--

OK, we're getting lost in the vagaries of English.

Here's a rule of thumb that I use. You _must_ store all fields when using 
atomic updates that are _not_ the destination of a copyField directive. If you 
don't, any data associated with a non-stored field that is _not_ the 
destination of a copyField directive will be lost.

How's that?

 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)

2014-04-25 Thread James Dyer (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981214#comment-13981214
 ] 

James Dyer commented on SOLR-5981:
--

The idea with the introduction of the DIHWriter interface was that users can 
have DIH bring documents into other environments than Solr.  But because DIH 
runs within Solr, it still passes the SolrWriter to DocBuilder.  Really this 
should have been refactored not to do this (see SOLR-3671).  This way you 
wouldn't need to be extending SolrWriter at all.

 Please change method visibility of getSolrWriter in DataImportHandler to 
 public (or at least protected)
 ---

 Key: SOLR-5981
 URL: https://issues.apache.org/jira/browse/SOLR-5981
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.0
 Environment: Linux 3.13.9-200.fc20.x86_64
 Solr 4.6.0
Reporter: Aaron LaBella
Assignee: Shawn Heisey
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-5981.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 I've been using the org.apache.solr.handler.dataimport.DataImportHandler for 
 a bit and it's an excellent model and architecture.  I'd like to extend the 
 usage of it to plugin my own DIHWriter, but, the code doesn't allow for it.  
 Please change ~line 227 in the DataImportHander class to be:
 public SolrWriter getSolrWriter
 instead of:
 private SolrWriter getSolrWriter
 or, at a minimum, protected, so that I can extend DataImportHandler and 
 override this method.
 Thank you *sincerely* in advance for the quick turn-around on this.  If the 
 change can be made in 4.6.0 and upstream, that'd be ideal.
 Thanks!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer

2014-04-25 Thread Ryan Ernst (JIRA)
Ryan Ernst created SOLR-6017:


 Summary: SimpleQParser uses index analyzer instead of query 
analyzer
 Key: SOLR-6017
 URL: https://issues.apache.org/jira/browse/SOLR-6017
 Project: Solr
  Issue Type: Bug
Reporter: Ryan Ernst


The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer().



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Kingston Duffie (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981230#comment-13981230
 ] 

Kingston Duffie commented on SOLR-6003:
---

Sounds unambiguous, but I want to be sure because of comments from others
on this thread.  Consider this hypothetical schema:

field A:  indexed=true, stored=true
field B:  indexed=true, stored=false

There are no copyFields at all.

With this schema, if I do an atomic update to field A on a given document,
say, {a : {set : newvalue}}, I will lose information in the index about
field B for that document.  Correct?





 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-6018) Solr DataImportHandler not finding dynamic fields

2014-04-25 Thread Aaron LaBella (JIRA)
Aaron LaBella created SOLR-6018:
---

 Summary: Solr DataImportHandler not finding dynamic fields
 Key: SOLR-6018
 URL: https://issues.apache.org/jira/browse/SOLR-6018
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.7
Reporter: Aaron LaBella
 Fix For: 4.9


There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields 
around ~line 643.  The logic currently says see if you can find the field from 
the schema, ie:

SchemaField sf = schema.getFieldOrNull(key);

and, if not found, go ask DIHConfiguration to find it:




--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields

2014-04-25 Thread Aaron LaBella (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron LaBella updated SOLR-6018:


Description: 
There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields 
around ~line 643.  The logic currently says see if you can find the field from 
the schema, ie:

SchemaField sf = schema.getFieldOrNull(key);

and, if not found, go ask DIHConfiguration to find it, ie:

sf = config.getSchemaField(key);

The latter call takes into account case-insensitivity, which is a big deal 
since some databases, ie: DB2, upper case all the resulting column names.  In 
order to not modify solr-core (ie: the match login in IndexSchema), I'm 
attaching a patch that makes DIHConfiguration apply the same case-insensitive 
logic to the DynamicFields.

Without this patch, dynamic fields will not be added to the index unless you 
declare them like this:

  dynamicField name=*_Stype=string   indexed=true stored=true 
/

(note the capital S)

which is in-consistent with what I believe to be solr schema conventions to 
have all the schema fields as lower-case.

Thanks.


  was:
There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields 
around ~line 643.  The logic currently says see if you can find the field from 
the schema, ie:

SchemaField sf = schema.getFieldOrNull(key);

and, if not found, go ask DIHConfiguration to find it:



 Solr DataImportHandler not finding dynamic fields
 -

 Key: SOLR-6018
 URL: https://issues.apache.org/jira/browse/SOLR-6018
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.7
Reporter: Aaron LaBella
 Fix For: 4.9


 There is an issue with 
 org.apache.solr.handler.dataimport.DocBuilder.addFields around ~line 643.  
 The logic currently says see if you can find the field from the schema, ie:
 SchemaField sf = schema.getFieldOrNull(key);
 and, if not found, go ask DIHConfiguration to find it, ie:
 sf = config.getSchemaField(key);
 The latter call takes into account case-insensitivity, which is a big deal 
 since some databases, ie: DB2, upper case all the resulting column names.  In 
 order to not modify solr-core (ie: the match login in IndexSchema), I'm 
 attaching a patch that makes DIHConfiguration apply the same case-insensitive 
 logic to the DynamicFields.
 Without this patch, dynamic fields will not be added to the index unless you 
 declare them like this:
   dynamicField name=*_Stype=string   indexed=true 
 stored=true /
 (note the capital S)
 which is in-consistent with what I believe to be solr schema conventions to 
 have all the schema fields as lower-case.
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields

2014-04-25 Thread Aaron LaBella (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron LaBella updated SOLR-6018:


Description: 
There is an issue with 
*org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
The logic currently says see if you can find the field from the schema, ie:

{code:title=DocBuilder.java|borderStyle=solid}
SchemaField sf = schema.getFieldOrNull(key);
{code}

and, if not found, go ask DIHConfiguration to find it, ie:

{code:title=DocBuilder.java|borderStyle=solid}
sf = config.getSchemaField(key);
{code}

The latter call takes into account case-insensitivity, which is a big deal 
since some databases, ie: DB2, upper case all the resulting column names.  In 
order to not modify solr-core (ie: the match login in IndexSchema), I'm 
attaching a patch that makes DIHConfiguration apply the same case-insensitive 
logic to the DynamicFields.

Without this patch, dynamic fields will not be added to the index unless you 
declare them like this:

{code:xml}
  dynamicField name=*_Stype=string   indexed=true stored=true 
/
{code}

(note the capital S)

which is in-consistent with what I believe to be solr schema conventions to 
have all the schema fields as lower-case.

Thanks.


  was:
There is an issue with org.apache.solr.handler.dataimport.DocBuilder.addFields 
around ~line 643.  The logic currently says see if you can find the field from 
the schema, ie:

SchemaField sf = schema.getFieldOrNull(key);

and, if not found, go ask DIHConfiguration to find it, ie:

sf = config.getSchemaField(key);

The latter call takes into account case-insensitivity, which is a big deal 
since some databases, ie: DB2, upper case all the resulting column names.  In 
order to not modify solr-core (ie: the match login in IndexSchema), I'm 
attaching a patch that makes DIHConfiguration apply the same case-insensitive 
logic to the DynamicFields.

Without this patch, dynamic fields will not be added to the index unless you 
declare them like this:

  dynamicField name=*_Stype=string   indexed=true stored=true 
/

(note the capital S)

which is in-consistent with what I believe to be solr schema conventions to 
have all the schema fields as lower-case.

Thanks.



 Solr DataImportHandler not finding dynamic fields
 -

 Key: SOLR-6018
 URL: https://issues.apache.org/jira/browse/SOLR-6018
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.7
Reporter: Aaron LaBella
 Fix For: 4.9

 Attachments: 0001-dih-config-check-for-dynamic-fields.patch


 There is an issue with 
 *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
 The logic currently says see if you can find the field from the schema, ie:
 {code:title=DocBuilder.java|borderStyle=solid}
 SchemaField sf = schema.getFieldOrNull(key);
 {code}
 and, if not found, go ask DIHConfiguration to find it, ie:
 {code:title=DocBuilder.java|borderStyle=solid}
 sf = config.getSchemaField(key);
 {code}
 The latter call takes into account case-insensitivity, which is a big deal 
 since some databases, ie: DB2, upper case all the resulting column names.  In 
 order to not modify solr-core (ie: the match login in IndexSchema), I'm 
 attaching a patch that makes DIHConfiguration apply the same case-insensitive 
 logic to the DynamicFields.
 Without this patch, dynamic fields will not be added to the index unless you 
 declare them like this:
 {code:xml}
   dynamicField name=*_Stype=string   indexed=true 
 stored=true /
 {code}
 (note the capital S)
 which is in-consistent with what I believe to be solr schema conventions to 
 have all the schema fields as lower-case.
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields

2014-04-25 Thread Aaron LaBella (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron LaBella updated SOLR-6018:


Description: 
There is an issue with 
*org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
The logic currently says see if you can find the field from the schema, ie:

{code:title=DocBuilder.java|borderStyle=solid}
SchemaField sf = schema.getFieldOrNull(key);
{code}

and, if not found, go ask DIHConfiguration to find it, ie:

{code:title=DocBuilder.java|borderStyle=solid}
sf = config.getSchemaField(key);
{code}

The latter call takes into account case-insensitivity, which is a big deal 
since some databases, ie: DB2, upper case all the resulting column names.  In 
order to not modify solr-core (ie: the match login in IndexSchema), I'm 
attaching a patch that makes DIHConfiguration apply the same case-insensitive 
logic to the DynamicFields.

Without this patch, dynamic fields will not be added to the index unless you 
declare them like this:

{code:xml}
  dynamicField name=*_S type=string indexed=true stored=true /
{code}

(note the capital S)

which is in-consistent with what I believe to be solr schema conventions to 
have all the schema fields as lower-case.

Thanks.


  was:
There is an issue with 
*org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
The logic currently says see if you can find the field from the schema, ie:

{code:title=DocBuilder.java|borderStyle=solid}
SchemaField sf = schema.getFieldOrNull(key);
{code}

and, if not found, go ask DIHConfiguration to find it, ie:

{code:title=DocBuilder.java|borderStyle=solid}
sf = config.getSchemaField(key);
{code}

The latter call takes into account case-insensitivity, which is a big deal 
since some databases, ie: DB2, upper case all the resulting column names.  In 
order to not modify solr-core (ie: the match login in IndexSchema), I'm 
attaching a patch that makes DIHConfiguration apply the same case-insensitive 
logic to the DynamicFields.

Without this patch, dynamic fields will not be added to the index unless you 
declare them like this:

{code:xml}
  dynamicField name=*_Stype=string   indexed=true stored=true 
/
{code}

(note the capital S)

which is in-consistent with what I believe to be solr schema conventions to 
have all the schema fields as lower-case.

Thanks.



 Solr DataImportHandler not finding dynamic fields
 -

 Key: SOLR-6018
 URL: https://issues.apache.org/jira/browse/SOLR-6018
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.7
Reporter: Aaron LaBella
 Fix For: 4.9

 Attachments: 0001-dih-config-check-for-dynamic-fields.patch


 There is an issue with 
 *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
 The logic currently says see if you can find the field from the schema, ie:
 {code:title=DocBuilder.java|borderStyle=solid}
 SchemaField sf = schema.getFieldOrNull(key);
 {code}
 and, if not found, go ask DIHConfiguration to find it, ie:
 {code:title=DocBuilder.java|borderStyle=solid}
 sf = config.getSchemaField(key);
 {code}
 The latter call takes into account case-insensitivity, which is a big deal 
 since some databases, ie: DB2, upper case all the resulting column names.  In 
 order to not modify solr-core (ie: the match login in IndexSchema), I'm 
 attaching a patch that makes DIHConfiguration apply the same case-insensitive 
 logic to the DynamicFields.
 Without this patch, dynamic fields will not be added to the index unless you 
 declare them like this:
 {code:xml}
   dynamicField name=*_S type=string indexed=true stored=true /
 {code}
 (note the capital S)
 which is in-consistent with what I believe to be solr schema conventions to 
 have all the schema fields as lower-case.
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields

2014-04-25 Thread Aaron LaBella (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron LaBella updated SOLR-6018:


Attachment: 0001-dih-config-check-for-dynamic-fields.patch

 Solr DataImportHandler not finding dynamic fields
 -

 Key: SOLR-6018
 URL: https://issues.apache.org/jira/browse/SOLR-6018
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.7
Reporter: Aaron LaBella
 Fix For: 4.9

 Attachments: 0001-dih-config-check-for-dynamic-fields.patch


 There is an issue with 
 *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
 The logic currently says see if you can find the field from the schema, ie:
 {code:title=DocBuilder.java|borderStyle=solid}
 SchemaField sf = schema.getFieldOrNull(key);
 {code}
 and, if not found, go ask DIHConfiguration to find it, ie:
 {code:title=DocBuilder.java|borderStyle=solid}
 sf = config.getSchemaField(key);
 {code}
 The latter call takes into account case-insensitivity, which is a big deal 
 since some databases, ie: DB2, upper case all the resulting column names.  In 
 order to not modify solr-core (ie: the match login in IndexSchema), I'm 
 attaching a patch that makes DIHConfiguration apply the same case-insensitive 
 logic to the DynamicFields.
 Without this patch, dynamic fields will not be added to the index unless you 
 declare them like this:
 {code:xml}
   dynamicField name=*_Stype=string   indexed=true 
 stored=true /
 {code}
 (note the capital S)
 which is in-consistent with what I believe to be solr schema conventions to 
 have all the schema fields as lower-case.
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)

2014-04-25 Thread Aaron LaBella (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981251#comment-13981251
 ] 

Aaron LaBella commented on SOLR-5981:
-

James, good point.  But, I think the natural place to start extending things is 
by extending the DataImportHandler so that you can take advantage of a request 
handler, ie:

requestHandler name=/dih 
class=org.apache.solr.handler.dataimport.DataImportHandler

Otherwise, you have to start doing all that work yourself.

 Please change method visibility of getSolrWriter in DataImportHandler to 
 public (or at least protected)
 ---

 Key: SOLR-5981
 URL: https://issues.apache.org/jira/browse/SOLR-5981
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.0
 Environment: Linux 3.13.9-200.fc20.x86_64
 Solr 4.6.0
Reporter: Aaron LaBella
Assignee: Shawn Heisey
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-5981.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 I've been using the org.apache.solr.handler.dataimport.DataImportHandler for 
 a bit and it's an excellent model and architecture.  I'd like to extend the 
 usage of it to plugin my own DIHWriter, but, the code doesn't allow for it.  
 Please change ~line 227 in the DataImportHander class to be:
 public SolrWriter getSolrWriter
 instead of:
 private SolrWriter getSolrWriter
 or, at a minimum, protected, so that I can extend DataImportHandler and 
 override this method.
 Thank you *sincerely* in advance for the quick turn-around on this.  If the 
 change can be made in 4.6.0 and upstream, that'd be ideal.
 Thanks!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6018) Solr DataImportHandler not finding dynamic fields

2014-04-25 Thread Aaron LaBella (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron LaBella updated SOLR-6018:


Description: 
There is an issue with 
*org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
The logic currently says see if you can find the field from the schema, ie:

{code:title=DocBuilder.java|borderStyle=solid}
SchemaField sf = schema.getFieldOrNull(key);
{code}

and, if not found, go ask DIHConfiguration to find it, ie:

{code:title=DocBuilder.java|borderStyle=solid}
sf = config.getSchemaField(key);
{code}

The latter call takes into account case-insensitivity, which is a big deal 
since some databases, ie: DB2, upper case all the resulting column names.  In 
order to not modify solr-core (ie: the match logic in IndexSchema), I'm 
attaching a patch that makes DIHConfiguration apply the same case-insensitive 
logic to the DynamicFields.

Without this patch, dynamic fields will not be added to the index unless you 
declare them like this:

{code:xml}
  dynamicField name=*_S type=string indexed=true stored=true /
{code}

(note the capital S)

which is in-consistent with what I believe to be solr schema conventions to 
have all the schema fields as lower-case.

Thanks.


  was:
There is an issue with 
*org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
The logic currently says see if you can find the field from the schema, ie:

{code:title=DocBuilder.java|borderStyle=solid}
SchemaField sf = schema.getFieldOrNull(key);
{code}

and, if not found, go ask DIHConfiguration to find it, ie:

{code:title=DocBuilder.java|borderStyle=solid}
sf = config.getSchemaField(key);
{code}

The latter call takes into account case-insensitivity, which is a big deal 
since some databases, ie: DB2, upper case all the resulting column names.  In 
order to not modify solr-core (ie: the match login in IndexSchema), I'm 
attaching a patch that makes DIHConfiguration apply the same case-insensitive 
logic to the DynamicFields.

Without this patch, dynamic fields will not be added to the index unless you 
declare them like this:

{code:xml}
  dynamicField name=*_S type=string indexed=true stored=true /
{code}

(note the capital S)

which is in-consistent with what I believe to be solr schema conventions to 
have all the schema fields as lower-case.

Thanks.



 Solr DataImportHandler not finding dynamic fields
 -

 Key: SOLR-6018
 URL: https://issues.apache.org/jira/browse/SOLR-6018
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.7
Reporter: Aaron LaBella
 Fix For: 4.9

 Attachments: 0001-dih-config-check-for-dynamic-fields.patch


 There is an issue with 
 *org.apache.solr.handler.dataimport.DocBuilder:addFields* around ~line 643.  
 The logic currently says see if you can find the field from the schema, ie:
 {code:title=DocBuilder.java|borderStyle=solid}
 SchemaField sf = schema.getFieldOrNull(key);
 {code}
 and, if not found, go ask DIHConfiguration to find it, ie:
 {code:title=DocBuilder.java|borderStyle=solid}
 sf = config.getSchemaField(key);
 {code}
 The latter call takes into account case-insensitivity, which is a big deal 
 since some databases, ie: DB2, upper case all the resulting column names.  In 
 order to not modify solr-core (ie: the match logic in IndexSchema), I'm 
 attaching a patch that makes DIHConfiguration apply the same case-insensitive 
 logic to the DynamicFields.
 Without this patch, dynamic fields will not be added to the index unless you 
 declare them like this:
 {code:xml}
   dynamicField name=*_S type=string indexed=true stored=true /
 {code}
 (note the capital S)
 which is in-consistent with what I believe to be solr schema conventions to 
 have all the schema fields as lower-case.
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer

2014-04-25 Thread Ryan Ernst (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Ernst updated SOLR-6017:
-

Attachment: SOLR-6017.patch

Patch with test.

 SimpleQParser uses index analyzer instead of query analyzer
 ---

 Key: SOLR-6017
 URL: https://issues.apache.org/jira/browse/SOLR-6017
 Project: Solr
  Issue Type: Bug
Reporter: Ryan Ernst
 Attachments: SOLR-6017.patch


 The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer().



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Shawn Heisey (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981265#comment-13981265
 ] 

Shawn Heisey commented on SOLR-6003:


bq. With this schema, if I do an atomic update to field A on a given document, 
say, {a : {set : newvalue}}, I will lose information in the index about field 
B for that document. Correct?

Correct.




 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5806) SolrCloud: UI link or script to remove node from clusterstate.json

2014-04-25 Thread Steven Bower (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981268#comment-13981268
 ] 

Steven Bower commented on SOLR-5806:


Would be a valuable API to have for both external cases like what is described 
and for the nodes themselves to decide they should go offline

 SolrCloud: UI link or script to remove node from clusterstate.json
 --

 Key: SOLR-5806
 URL: https://issues.apache.org/jira/browse/SOLR-5806
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.7
Reporter: Gregg Donovan
Priority: Minor

 In cases of partial failure where a node is still connected to ZooKeeper but 
 failing -- e.g. bad disk, bad memory, etc. -- it would be nice to have a 
 quick UI link or command-line script to remove the node from 
 clusterstate.json quickly.
 We've had partial failures where we couldn't SSH into the box but the VM was 
 still running and connected to ZooKeeper. In these cases, we've had to power 
 the machine down from the 
 [ILO|http://en.wikipedia.org/wiki/HP_Integrated_Lights-Out] in order to get 
 it out of clusterstate.json. 
 Having something handier in such outages would be great. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Hoss Man (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981280#comment-13981280
 ] 

Hoss Man commented on SOLR-6003:


bq. With this schema, if I do an atomic update to field A on a given document, 
say, {a : {set : newvalue}}, I will lose information in the index about field 
B for that document. Correct?

Correct: but that may have been your intent when designing that schema.  You 
may have set it up that way precisely because you know that anytime you update 
a document you will either be explicitly supplying a replacement value for 
field b, or deliberately excluding a value for field b so that the current 
field goes away.

This is what i was refering to in my earlier comment...

bq. ...or the use cases i've seen in the wild where users deliberately have 
non-stored fields (to save space) that they might replace with new values on 
atomic update, or they just leave the field out of the update as a way to 
remove the value. (a deliberate choice of one or the other is made by their 
update client on every update)

just because an atomic update causes information to be removed from a 
non-stored field, doesn't mean that removal isn't a deliberate part o the use 
case

---

bq. This specific Jira should focus on the specific issue of increment for a 
non-stored field, and append to a non-stored multivalued field. Clearly this 
case should produce an exception since it can't possibly do anything reasonable 
since it needs to access the previous value before applying the increment or 
append.

Agreed - we can't, in general, make assumptions about failing _any_ atomic 
update just because some fields in the schema aren't stored -- but i don't see 
any reason why we can't fail in some _specific_ cases of things like  inc or 
remove directly on non-stored fields. (I'm surprised we aren't already)

 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Kingston Duffie (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981295#comment-13981295
 ] 

Kingston Duffie commented on SOLR-6003:
---

Hoss Man, you make an excellent point that I was simply failing to think
about -- the case that someone may be fine if their existing indexed
content is lost for a non-stored field -- typically because they will be
providing new information, if necessary, for that field and others.  Thanks
for helping me understand.  With that in mind, I'm fine in closing this
issue -- while also even more strongly looking forward to improvements in
this area in future.





 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting

2014-04-25 Thread Mark Miller (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-2894:
--

Attachment: SOLR-2894.patch

I have not worked on faceting code in the past, so this is really not my area.

However, here is a patch I just worked up to apply against 5x. I had to make 
some small changes - DateField is deprecated and there was an ndate field in 
the tests that could not be found. I removed it in this patch. I also fixed a 
few issues around licenses and formatting - this patch passes precommit except 
for a nocommit at the DateField change - someone should review if that change 
has any other ramifications. All tests pass.

This code does touch some existing faceting code in a way that demands a deeper 
review I think, but until I have a lot more time, I'm not the man for that job. 
Perhaps [~hossman_luc...@fucit.org]?

 Implement distributed pivot faceting
 

 Key: SOLR-2894
 URL: https://issues.apache.org/jira/browse/SOLR-2894
 Project: Solr
  Issue Type: Improvement
Reporter: Erik Hatcher
 Fix For: 4.9, 5.0

 Attachments: SOLR-2894-reworked.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, dateToObject.patch


 Following up on SOLR-792, pivot faceting currently only supports 
 undistributed mode.  Distributed pivot faceting needs to be implemented.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Assigned] (SOLR-6015) managed synonyms don't gracefully handle lowercasing

2014-04-25 Thread Timothy Potter (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Potter reassigned SOLR-6015:


Assignee: Timothy Potter

 managed synonyms don't gracefully handle lowercasing
 

 Key: SOLR-6015
 URL: https://issues.apache.org/jira/browse/SOLR-6015
 Project: Solr
  Issue Type: Bug
Reporter: Yonik Seeley
Assignee: Timothy Potter

 I've been having bad luck testing new functionallity lately - the first thing 
 I try never works ;-)
 {code}
 /opt/code/lusolr48/solr/example/solr/collection1/conf$ curl -XPUT 
 http://localhost:8983/solrsis/synonyms/english;   -H 
 'Content-type:application/json'   --data-binary '{MB:[MiB,Megabyte]}'
 {
   responseHeader:{
 status:500,
 QTime:3},
   error:{
 msg:Bad Request,
 trace:Bad Request (400) - Unsupported value null for mb; expected 
 single value or a JSON array!\n
 [...]
 {code}
 I finally figured out that if I lowercased MB to mb, then it works as 
 expected.
 Also, it looks like because ignoreCase is true in the initArgs, everything is 
 *stored* as lower case in the managed map (losing information).  Ideally case 
 would be preserved in the synonym file (esp if one wanted to change 
 ignoreCase either way).  I imagine stopwords may have the same issue, but I 
 haven't checked.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Closed] (SOLR-6003) JSON Update increment field with non-stored fields causes subtle problems

2014-04-25 Thread Erick Erickson (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson closed SOLR-6003.


Resolution: Not a Problem

Closing as per Kingston's OK.

 JSON Update increment field with non-stored fields causes subtle problems
 -

 Key: SOLR-6003
 URL: https://issues.apache.org/jira/browse/SOLR-6003
 Project: Solr
  Issue Type: Bug
  Components: update
Affects Versions: 4.7.1
Reporter: Kingston Duffie

 In our application we have large multi-field documents.  We occasionally need 
 to increment one of the numeric fields or add a value to a multi-value text 
 field.  This appears to work correctly using JSON update.  But later we 
 discovered that documents were disappearing from search results and 
 eventually found the documentation that indicates that to use field 
 modification you must store all fields of the document.
 Perhaps you will argue that you need to impose this restriction -- which I 
 would hope could be overcome because of the cost of us having to store all 
 fields.  But in any case, it would be better for others if you could return 
 an error if someone tries to update a field on documents with non-stored 
 fields.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer

2014-04-25 Thread Erick Erickson (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981338#comment-13981338
 ] 

Erick Erickson commented on SOLR-6017:
--

Gaah! What do people think about refactoring getAnalyzer() to 
getIndexAnalyzer()? This is trappy...

Actually, since it's a public method I suppose we'd have to deprecate it and 
just have it call a new getIndexAnalyzer(). There's only 10 or so places it's 
called in Solr.

At least that way an IDE's autocomplete would give some clue that there even 
_were_ two methods

 SimpleQParser uses index analyzer instead of query analyzer
 ---

 Key: SOLR-6017
 URL: https://issues.apache.org/jira/browse/SOLR-6017
 Project: Solr
  Issue Type: Bug
Reporter: Ryan Ernst
 Attachments: SOLR-6017.patch


 The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer().



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5145) wait w/o notify situation happening on shutdown in CoreContainer/CloserThread/SolrCores.modifyLock

2014-04-25 Thread Timothy Potter (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981400#comment-13981400
 ] 

Timothy Potter commented on SOLR-5145:
--

I'm almost certain this is because somewhere in the code for this test, the 
core doesn't get closed. Look for instances of cc.getCore(name) outside of the 
try with resources block, i.e. should be doing:

  try (SolrCore core = cc.getCore(coreName)) {
  // code
  }

 wait w/o notify situation happening on shutdown in 
 CoreContainer/CloserThread/SolrCores.modifyLock
 --

 Key: SOLR-5145
 URL: https://issues.apache.org/jira/browse/SOLR-5145
 Project: Solr
  Issue Type: Bug
Reporter: Hoss Man

 Working on SOLR-4952 i was running tests on the 4x branch and noticed 
 TestManagedSchema.testPersistUniqueKey stalled for 1623s -- but nothing baout 
 hte situation seems to be specific to that test.
 I'll attache a threaddump, but the key things i note...
 * TestHarness is trying to shutdown the CoreContainer
 * CoreContainer.shutdown(CoreContainer.java:358) is waiting on 
 0xf690b880 (a org.apache.solr.core.CloserThread)
 * CloserThread.run(CoreContainer.java:961) is waiting on 0xe068d718 
 (a java.lang.Object)
 ** that's aparently SolrCores.modifyLock
 * a searcherExecutor-46-thread-1 is still alive and parking to wait for  
 0xf66ff860 (a 
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
 * no other threads seem to be doing anything -- so as far as i can tell 
 nothing is ever going to notify on that modifyLock



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-5331) nested SpanNearQuery with repeating groups does not find match

2014-04-25 Thread Michael Sander (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981405#comment-13981405
 ] 

Michael Sander commented on LUCENE-5331:


This bug is actually quite important to my app.  I will pay whoever fixes it 
$500 for their efforts or donate the same to the Lucene or Apache foundations.  
Feel free to message me for more details. Offer expires July 2014.

 nested SpanNearQuery with repeating groups does not find match
 --

 Key: LUCENE-5331
 URL: https://issues.apache.org/jira/browse/LUCENE-5331
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Jerry Zhou
 Attachments: NestedSpanNearTest.java


 Nested spanNear queries do not work in some cases when repeating groups are 
 in the query.
 Test case is attached ...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Comment Edited] (LUCENE-5331) nested SpanNearQuery with repeating groups does not find match

2014-04-25 Thread Michael Sander (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981405#comment-13981405
 ] 

Michael Sander edited comment on LUCENE-5331 at 4/25/14 6:23 PM:
-

This bug is actually quite important to my app.  I will pay whoever fixes it 
$500 USD for their efforts or, if they prefer, donate the same to the Lucene or 
Apache foundations.  Feel free to message me for more details. Offer expires 
July 2014.


was (Author: speedplane):
This bug is actually quite important to my app.  I will pay whoever fixes it 
$500 for their efforts or donate the same to the Lucene or Apache foundations.  
Feel free to message me for more details. Offer expires July 2014.

 nested SpanNearQuery with repeating groups does not find match
 --

 Key: LUCENE-5331
 URL: https://issues.apache.org/jira/browse/LUCENE-5331
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Jerry Zhou
 Attachments: NestedSpanNearTest.java


 Nested spanNear queries do not work in some cases when repeating groups are 
 in the query.
 Test case is attached ...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Comment Edited] (LUCENE-5331) nested SpanNearQuery with repeating groups does not find match

2014-04-25 Thread Michael Sander (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981405#comment-13981405
 ] 

Michael Sander edited comment on LUCENE-5331 at 4/25/14 6:24 PM:
-

This bug is actually quite important to my app.  I will pay whoever fixes it 
$500 USD for their efforts or, if they prefer, donate the same to the Lucene or 
Apache foundations.  Feel free to message me for more details. Offer expires 
July 2014.

And for clarification, I am referring to the bug where a nested span returns 
more results than two separate spans, identified in my comment above.


was (Author: speedplane):
This bug is actually quite important to my app.  I will pay whoever fixes it 
$500 USD for their efforts or, if they prefer, donate the same to the Lucene or 
Apache foundations.  Feel free to message me for more details. Offer expires 
July 2014.

 nested SpanNearQuery with repeating groups does not find match
 --

 Key: LUCENE-5331
 URL: https://issues.apache.org/jira/browse/LUCENE-5331
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Jerry Zhou
 Attachments: NestedSpanNearTest.java


 Nested spanNear queries do not work in some cases when repeating groups are 
 in the query.
 Test case is attached ...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-6019) Managed schema file does not show up in the Files UI

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)
Shalin Shekhar Mangar created SOLR-6019:
---

 Summary: Managed schema file does not show up in the Files UI
 Key: SOLR-6019
 URL: https://issues.apache.org/jira/browse/SOLR-6019
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis, web gui
Affects Versions: 4.7.2, 4.8
Reporter: Shalin Shekhar Mangar


When running with the schema-less example, I noticed that the managed-schema 
file does not show in the Files section of the Admin UI. This can be 
confusing for a user. To make sure it was not a caching issue on the browser, I 
closed and opened the UI again in a new tab. I also restarted Solr and still 
the managed-schema is not visible in the Files section. Interestingly, the 
schema.xml.bak does show up. A screenshot of the UI is attached.

It is possible that this bug affects other managed resources as well such as 
synonyms but I haven't tested that yet.

The schema browser works fine though.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6019) Managed schema file does not show up in the Files UI

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-6019:


Attachment: 6019-missing-managed-schema.png

 Managed schema file does not show up in the Files UI
 --

 Key: SOLR-6019
 URL: https://issues.apache.org/jira/browse/SOLR-6019
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis, web gui
Affects Versions: 4.7.2, 4.8
Reporter: Shalin Shekhar Mangar
 Attachments: 6019-missing-managed-schema.png


 When running with the schema-less example, I noticed that the managed-schema 
 file does not show in the Files section of the Admin UI. This can be 
 confusing for a user. To make sure it was not a caching issue on the browser, 
 I closed and opened the UI again in a new tab. I also restarted Solr and 
 still the managed-schema is not visible in the Files section. Interestingly, 
 the schema.xml.bak does show up. A screenshot of the UI is attached.
 It is possible that this bug affects other managed resources as well such as 
 synonyms but I haven't tested that yet.
 The schema browser works fine though.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-6016) Failure indexing exampledocs with example-schemaless mode

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-6016:


Description: 
Steps to reproduce:
# cd example; java -Dsolr.solr.home=example-schemaless/solr -jar start.jar
# cd exampledocs; java -jar post.jar *.xml

Output from post.jar
{code}
Posting files to base url http://localhost:8983/solr/update using content-type 
application/xml..
POSTing file gb18030-example.xml
POSTing file hd.xml
POSTing file ipod_other.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file ipod_video.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file manufacturers.xml
POSTing file mem.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file money.xml
POSTing file monitor2.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file monitor.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file mp500.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file sd500.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file solr.xml
POSTing file utf8-example.xml
POSTing file vidcard.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
14 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/update..
Time spent: 0:00:00.401
{code}

Exceptions in Solr (I am pasting just one of them):
{code}
5105 [qtp697879466-14] ERROR org.apache.solr.core.SolrCore  – 
org.apache.solr.common.SolrException: ERROR: [doc=EN7800GTX/2DHTV/256M] Error 
adding field 'price'='479.95' msg=For input string: 479.95
at 
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167)
at 
org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:234)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160)
at 
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
at 
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
..
Caused by: java.lang.NumberFormatException: For input string: 479.95
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:441)
at java.lang.Long.parseLong(Long.java:483)
at org.apache.solr.schema.TrieField.createField(TrieField.java:609)
at org.apache.solr.schema.TrieField.createFields(TrieField.java:660)
{code}

The full solr.log is attached.

I understand why these errors occur but since we ship example data with Solr to 
demonstrate our core features, I expect that indexing exampledocs should work 
without errors.

  was:
Steps to reproduce:
# cd example; java -Dsolr.solr.home=example-schemaless/solr -jar start.jar
# cd ../exampledocs; java -jar post.jar *.xml

Output from post.jar
{code}
Posting files to base url http://localhost:8983/solr/update using content-type 
application/xml..
POSTing file gb18030-example.xml
POSTing file hd.xml
POSTing file ipod_other.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/update
POSTing file ipod_video.xml
SimplePostTool: WARNING: Solr returned an error #400 Bad Request

[jira] [Created] (SOLR-6020) Auto-generate a unique key in schema-less mode if data does not have an id field

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)
Shalin Shekhar Mangar created SOLR-6020:
---

 Summary: Auto-generate a unique key in schema-less mode if data 
does not have an id field
 Key: SOLR-6020
 URL: https://issues.apache.org/jira/browse/SOLR-6020
 Project: Solr
  Issue Type: Improvement
  Components: Schema and Analysis
Reporter: Shalin Shekhar Mangar


Currently it is not possible to use the schema-less example if my data does not 
have an id field.

I was indexing data where the unique field name was url in schema-less mode. 
This requires one to first change unique key name in the schema and then start 
solr and then index docs. If one had already started solr, one'd first need to 
remove managed-schema, rename schema.xml.bak to schema.xml and then make the 
necessary changes in schema.xml. I don't think we should fail on such simple 
things.

Here's what I propose:
# We remove id and uniqueKey from the managed schema example
# If there's a field named id in the document,  we use that as the uniqueKey
# Else we fallback on generating a UUID or a signature field via an update 
processor and store it as the unique key field. We can name it as id or _id
# But if a uniqueKey is already present in original schema.xml then we should 
expect the incoming data to have that field and we should preserve the current 
behavior of failing loudly.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6020) Auto-generate a unique key in schema-less mode if data does not have an id field

2014-04-25 Thread Hoss Man (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981493#comment-13981493
 ] 

Hoss Man commented on SOLR-6020:


Wouldn't the simplest solution in this case be...

* leave uniqueKey (id,string) in 
example-schemaless/solr/collection1/conf/managed-schema
* add UUIDUpdateProcessorFactory (id) to 
example-schemaless/solr/collection1/conf/solconfig.xml ?

UUIDUpdateProcessorFactory will already do the right thing and not generate a 
new ID if the document being added already has one.

 Auto-generate a unique key in schema-less mode if data does not have an id 
 field
 --

 Key: SOLR-6020
 URL: https://issues.apache.org/jira/browse/SOLR-6020
 Project: Solr
  Issue Type: Improvement
  Components: Schema and Analysis
Reporter: Shalin Shekhar Mangar

 Currently it is not possible to use the schema-less example if my data does 
 not have an id field.
 I was indexing data where the unique field name was url in schema-less 
 mode. This requires one to first change unique key name in the schema and 
 then start solr and then index docs. If one had already started solr, one'd 
 first need to remove managed-schema, rename schema.xml.bak to schema.xml and 
 then make the necessary changes in schema.xml. I don't think we should fail 
 on such simple things.
 Here's what I propose:
 # We remove id and uniqueKey from the managed schema example
 # If there's a field named id in the document,  we use that as the uniqueKey
 # Else we fallback on generating a UUID or a signature field via an update 
 processor and store it as the unique key field. We can name it as id or 
 _id
 # But if a uniqueKey is already present in original schema.xml then we should 
 expect the incoming data to have that field and we should preserve the 
 current behavior of failing loudly.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6020) Auto-generate a unique key in schema-less mode if data does not have an id field

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981495#comment-13981495
 ] 

Shalin Shekhar Mangar commented on SOLR-6020:
-

+1

That's even better. I wasn't aware that UUIDUpdateProcessor can do that.

 Auto-generate a unique key in schema-less mode if data does not have an id 
 field
 --

 Key: SOLR-6020
 URL: https://issues.apache.org/jira/browse/SOLR-6020
 Project: Solr
  Issue Type: Improvement
  Components: Schema and Analysis
Reporter: Shalin Shekhar Mangar

 Currently it is not possible to use the schema-less example if my data does 
 not have an id field.
 I was indexing data where the unique field name was url in schema-less 
 mode. This requires one to first change unique key name in the schema and 
 then start solr and then index docs. If one had already started solr, one'd 
 first need to remove managed-schema, rename schema.xml.bak to schema.xml and 
 then make the necessary changes in schema.xml. I don't think we should fail 
 on such simple things.
 Here's what I propose:
 # We remove id and uniqueKey from the managed schema example
 # If there's a field named id in the document,  we use that as the uniqueKey
 # Else we fallback on generating a UUID or a signature field via an update 
 processor and store it as the unique key field. We can name it as id or 
 _id
 # But if a uniqueKey is already present in original schema.xml then we should 
 expect the incoming data to have that field and we should preserve the 
 current behavior of failing loudly.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6013) Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981528#comment-13981528
 ] 

Shalin Shekhar Mangar commented on SOLR-6013:
-

Thanks Aaron. Some comments:
# DateFormatEvaluator methods such as evaluateWrapper, evaluateString, 
parseMathString, resolveMapper and getDateMathParser have no business being 
public but we can make them protected if you want. All of them should be marked 
as experimental API.
# Evaluator.parseParams should be protected not public.
# I don't like that we are creating a method such as getVariableWrapper. These 
things are not supposed to be pluggable and it should definitely not be public. 
We can mark it as protected along with a javadoc warning saying that this is 
experimental API. If I were writing Evaluator today, I'd just use a Callable 
instead.

On an unrelated note, I wonder why we cache all available locales and 
timezones. If looking up locale/timezone is expensive then it can be done 
as-needed. I'll create an issue.

 Fix method visibility of Evaluator, refactor DateFormatEvaluator for 
 extensibility
 --

 Key: SOLR-6013
 URL: https://issues.apache.org/jira/browse/SOLR-6013
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.7
Reporter: Aaron LaBella
 Fix For: 4.8

 Attachments: 0001-add-getters-for-datemathparser.patch, 
 0001-change-method-variable-visibility-and-refactor-for-extensibility.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 This is similar to issue 5981, the Evaluator class is declared as abstract, 
 yet the parseParams method is package private?  Surely this is an oversight, 
 as I wouldn't expect everyone writing their own evaluators to have to deal 
 with parsing the parameters.
 Similarly, I needed to refactor DateFormatEvaluator because I need to do some 
 custom date math/parsing and it wasn't written in a way that I can extend it.
 Please review/apply my attached patch to the next version of Solr, ie: 4.8 or 
 4.9 if I must wait.
 Thanks!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-6021) Always persist router.field in cluster state so CloudSolrServer can route documents correctly

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)
Shalin Shekhar Mangar created SOLR-6021:
---

 Summary: Always persist router.field in cluster state so 
CloudSolrServer can route documents correctly
 Key: SOLR-6021
 URL: https://issues.apache.org/jira/browse/SOLR-6021
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Shalin Shekhar Mangar


CloudSolrServer has idField as id which is used for hashing and distributing 
documents. There is a setter to change it as well.

IMO, we should use the correct uniqueKey automatically. I propose that we start 
storing router.field always in cluster state and set it to the uniqueKey field 
name by default. Then CloudSolrServer would not need to assume an id field by 
default.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6021) Always persist router.field in cluster state so CloudSolrServer can route documents correctly

2014-04-25 Thread Mark Miller (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981592#comment-13981592
 ] 

Mark Miller commented on SOLR-6021:
---

+1 - I had been thinking about working on this not long ago.

 Always persist router.field in cluster state so CloudSolrServer can route 
 documents correctly
 -

 Key: SOLR-6021
 URL: https://issues.apache.org/jira/browse/SOLR-6021
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Shalin Shekhar Mangar

 CloudSolrServer has idField as id which is used for hashing and 
 distributing documents. There is a setter to change it as well.
 IMO, we should use the correct uniqueKey automatically. I propose that we 
 start storing router.field always in cluster state and set it to the 
 uniqueKey field name by default. Then CloudSolrServer would not need to 
 assume an id field by default.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6020) Auto-generate a unique key in schema-less mode if data does not have an id field

2014-04-25 Thread Hoss Man (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981611#comment-13981611
 ] 

Hoss Man commented on SOLR-6020:


A related improvement that might be easy: change UUIDUpdateProcessorFactory so 
that if no fieldName is configured, it defaults to the uniqueKey field in the 
schema (if the schema has one - else error just like it does right now if you 
forget to configure the fieldName on the processor)


 Auto-generate a unique key in schema-less mode if data does not have an id 
 field
 --

 Key: SOLR-6020
 URL: https://issues.apache.org/jira/browse/SOLR-6020
 Project: Solr
  Issue Type: Improvement
  Components: Schema and Analysis
Reporter: Shalin Shekhar Mangar

 Currently it is not possible to use the schema-less example if my data does 
 not have an id field.
 I was indexing data where the unique field name was url in schema-less 
 mode. This requires one to first change unique key name in the schema and 
 then start solr and then index docs. If one had already started solr, one'd 
 first need to remove managed-schema, rename schema.xml.bak to schema.xml and 
 then make the necessary changes in schema.xml. I don't think we should fail 
 on such simple things.
 Here's what I propose:
 # We remove id and uniqueKey from the managed schema example
 # If there's a field named id in the document,  we use that as the uniqueKey
 # Else we fallback on generating a UUID or a signature field via an update 
 processor and store it as the unique key field. We can name it as id or 
 _id
 # But if a uniqueKey is already present in original schema.xml then we should 
 expect the incoming data to have that field and we should preserve the 
 current behavior of failing loudly.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-5622) Fail tests if they print, and tests.verbose is not set

2014-04-25 Thread Robert Muir (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981625#comment-13981625
 ] 

Robert Muir commented on LUCENE-5622:
-

+1 to that idea. It at least prevents things from getting out of control, which 
is the most important.

 Fail tests if they print, and tests.verbose is not set
 --

 Key: LUCENE-5622
 URL: https://issues.apache.org/jira/browse/LUCENE-5622
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
Assignee: Dawid Weiss
 Attachments: LUCENE-5622.patch, LUCENE-5622.patch, LUCENE-5622.patch, 
 LUCENE-5622.patch


 Some tests print so much stuff they are now undebuggable (see LUCENE-5612).
 I think its bad that the testrunner hides this stuff, we used to stay on top 
 of it. Instead, whne tests.verbose is false, we should install a printstreams 
 (system.out/err) that fail the test instantly because they are noisy. 
 This will ensure that our tests don't go out of control.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: VOTE: RC0 Release apache-solr-ref-guide-4.8.pdf

2014-04-25 Thread Chris Hostetter

: 3. There are a bunch of pages with text in a much larger font size, 
: mixed with text in regularly-sized fonts.  I noticed this on pages 71, 
: 85, 86, 99, 100, 118, 120, 313-317, 338, and 345.  AFAICT, these all 
: correspond to very recent modifications.  I don’t remember seeing this 
: problem at all in the online HTML view of this content.  (I haven’t 
: checked yet to see if that’s still the case.)

FWIW; As far as sarowe and i can figure out, these are the result of 
cut/pasting text from ohter documents (or web pages) in which case 
Confluence seems to preserve information about hte original formatting 
(notably the font-size) but doesn't show this formatting in hte web view 
or give any indication of it in the editor -- it is however used when 
exporting to the PDF.

We think we've got them all, and i'll be putting out a new RC soon.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer

2014-04-25 Thread Ryan Ernst (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981631#comment-13981631
 ] 

Ryan Ernst commented on SOLR-6017:
--

bq. What do people think about refactoring getAnalyzer() to getIndexAnalyzer()? 
This is trappy...
I completely agree.  I created SOLR-6022 to address that.

 SimpleQParser uses index analyzer instead of query analyzer
 ---

 Key: SOLR-6017
 URL: https://issues.apache.org/jira/browse/SOLR-6017
 Project: Solr
  Issue Type: Bug
Reporter: Ryan Ernst
 Attachments: SOLR-6017.patch


 The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer().



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-6022) Rename getAnalyzer to getIndexAnalyzer

2014-04-25 Thread Ryan Ernst (JIRA)
Ryan Ernst created SOLR-6022:


 Summary: Rename getAnalyzer to getIndexAnalyzer
 Key: SOLR-6022
 URL: https://issues.apache.org/jira/browse/SOLR-6022
 Project: Solr
  Issue Type: Improvement
Reporter: Ryan Ernst


We have separate index/query analyzer chains, but the access methods for the 
analyzers do not match up with the names.  This can lead to unknowingly using 
the wrong analyzer chain (as it did in SOLR-6017).  We should do this renaming 
in trunk, and deprecate the old getAnalyzer function in 4x.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Assigned] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer

2014-04-25 Thread Ryan Ernst (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Ernst reassigned SOLR-6017:


Assignee: Ryan Ernst

 SimpleQParser uses index analyzer instead of query analyzer
 ---

 Key: SOLR-6017
 URL: https://issues.apache.org/jira/browse/SOLR-6017
 Project: Solr
  Issue Type: Bug
Reporter: Ryan Ernst
Assignee: Ryan Ernst
 Attachments: SOLR-6017.patch


 The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer().



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



VOTE: RC1 Release apache-solr-ref-guide-4.8.pdf

2014-04-25 Thread Chris Hostetter


(Note: cross posted to general, please confine replies to dev@lucene)

Please VOTE to release the following RC1 as apache-solr-ref-guide-4.8.pdf ...

https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-4.8-RC1


The notes I previously mentioned regarding RC0 apply to this RC as well...

1) Due to a known bug in confluence, the PDFs it generates are much bigger 
then they should be.  This bug has been fixed in the latest version of 
confluence, but cwiki.apache.rog has not yet been updated.  For that 
reason, I have manually run a small tool against the PDF to fix the size 
(see SOLR-5819).  The first time i tried this approach, it inadvertantly 
removed the Index (aka: Table of Contents, or Bookmarks depending on 
what PDF reader client you use).  I've already fixed this, but if you 
notice anything else unusual about this PDF compared to previous versions 
please speak up so we can see if it's a result of this post-processing and 
try to fix it.


2) This is the first ref guide release where we've started using a special 
confluence macro for any lucene/solr javadoc links.  The up side is that 
all javadoc links in this 4.8 ref guide will now correctly point to the 
4.8 javadocs on lucene.apache.org -- the down side is that this means none 
of those links currently work, since the 4.8 code release is still ongoing 
and the website has not yet been updated.


Because of #2, I intend to leave this ref guide vote open until the 4.8 
code release is final - that way we won't officially be releasing this doc 
until the 4.8 javadocs are uploaded and all the links work properly.




-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-5495) Recovery strategy for leader partitioned from replica case.

2014-04-25 Thread Timothy Potter (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Potter updated SOLR-5495:
-

Attachment: SOLR-5495.patch

Here's an updated patch that builds upon the previous one, same basic approach 
of leader-initiated recovery but with some added coordination between the 
leader and partitioned replica using a znode: 
/collections/collection/leader_initiated_recovery/shard/replicaCoreName 
(see ZkController.getLeaderInitiatedRecoveryZnodePath).

The basic idea here is in addition to the leader marking the replica down, a 
separate znode is used to track the replica's transition from down - 
recovering - active. So the leader marks the replica as down (which removes it 
from participating in queries and update requests) and also creates this 
special znode. When the replica finally gets the you need to recover command 
from the leader, it changes the value of this znode to recovering. When 
recovery succeeds, the replica deletes the znode as it's no longer needed. If 
the leader, while trying to send the recovery command (see 
LeaderInitiatedRecoveryThread), sees the replica as being active but the 
znode wasn't ack'd, then the leader can set the state to down again. As stated 
before, I don't see where the replica would do this, but if it happens, we now 
have a better way to handle it. Bottom line is with this special znode, the 
replica cannot stay in the active state until it acks the leader's command to 
recover by transitioning the znode appropriately.

The special znode is also useful if the nagging leader fails before the bad 
replica receives the message. The idea here is that the new leader can query ZK 
for any of these leader-initiated-recovery znodes for its shard and if there 
are any in the down state, then it can start up the nag loop for each bad 
replica; a znode in the down state means the replica hasn't received the 
recovery command yet (see: 
ElectionContext$ShardLeaderElectionContext.startLeaderInitiatedRecoveryOnReplicas).

There is a unit test that covers the leader failing over to a new leader and 
resuming the nag loop on the downed replica. There's one area where I'm not 
100% sure if it is correct yet ... in the shouldIBeLeader method in 
ShardLeaderElectionContext, I check to see if a previous leader marked this 
core down and if so, return false to indicate this node should not be the 
leader. I think this works OK for RF=3 but I'm worried about RF=2 situations 
where this check prevents a leader from being elected. The main idea behind 
this check is that if the leader forces the shard state to down, the 
core.getCoreDescriptor().getCloudDescriptor().getLastPublished() method can 
still return active so I needed this additional check on the znode. I suppose 
we could try to update the lastPublished state when it changes but didn't see 
how to go about that? (or if that was even a good idea).

Another area where I'm not 100% sold on is the 10-minute max wait and then 
timeout loop in the LeaderInitiatedRecoveryThread. 10 mins is arbitrary but it 
seems like it shouldn't just run forever. One idea I had was to use JMX to 
raise some event / notification to allow monitoring tools to alert ops team of 
this issue. Curious if there's anything else in SolrCloud related to notifying 
of issues that need ops attention?

Lastly, I did give some thought to a self-initiating recovery approach where 
the trying to recover loop runs on the replica itself as that is more immune 
to leader changes and there's already a recover retry loop in place via the 
RecoveryStrategy thread. As I understand it, a self-initiating approach would 
work something like:

1) leader receives error when trying to forward update request to replica
2) leader marks replica as down in ZK
3) replica receives state change notification (at some point), replica must 
iterate over all cores hosted on that node looking for cores marked as down
4) for each down core on the node found in step 3, try recovering in a loop

This is all straight-forward to implement. However, the main problem with this 
approach is in step 4, when starting recovery, the replica updates its state to 
recovering in ZK immediately. When a replica is recovering the leader still 
tries to forward updates to it (the updates get stashed in the tlog until 
recovery is complete). This works in normal circumstances because the replica 
assumes there is no partition between it and the leader so it's OK to go into 
the recovering state and continue receiving updates. The problem here though is 
the network may still be partitioned, so the leader keeps trying to forward 
docs and receiving errors. From the leader's perspective, we're right back at 
step 1 above.

Of course, it would be possible to introduce a new state that would prevent the 
leader from sending updates while the replica sorted itself out, but I'm 
hesitant to introduce a new 

[jira] [Commented] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer

2014-04-25 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981698#comment-13981698
 ] 

ASF subversion and git services commented on SOLR-6017:
---

Commit 1590166 from [~rjernst] in branch 'dev/trunk'
[ https://svn.apache.org/r1590166 ]

SOLR-6017: Fix SimpleQParser to use query analyzer instead of index analyzer

 SimpleQParser uses index analyzer instead of query analyzer
 ---

 Key: SOLR-6017
 URL: https://issues.apache.org/jira/browse/SOLR-6017
 Project: Solr
  Issue Type: Bug
Reporter: Ryan Ernst
Assignee: Ryan Ernst
 Attachments: SOLR-6017.patch


 The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer().



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer

2014-04-25 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981716#comment-13981716
 ] 

ASF subversion and git services commented on SOLR-6017:
---

Commit 1590176 from [~rjernst] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1590176 ]

SOLR-6017: Fix SimpleQParser to use query analyzer instead of index analyzer

 SimpleQParser uses index analyzer instead of query analyzer
 ---

 Key: SOLR-6017
 URL: https://issues.apache.org/jira/browse/SOLR-6017
 Project: Solr
  Issue Type: Bug
Reporter: Ryan Ernst
Assignee: Ryan Ernst
 Attachments: SOLR-6017.patch


 The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer().



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (SOLR-6017) SimpleQParser uses index analyzer instead of query analyzer

2014-04-25 Thread Ryan Ernst (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Ernst resolved SOLR-6017.
--

   Resolution: Fixed
Fix Version/s: 5.0
   4.9

 SimpleQParser uses index analyzer instead of query analyzer
 ---

 Key: SOLR-6017
 URL: https://issues.apache.org/jira/browse/SOLR-6017
 Project: Solr
  Issue Type: Bug
Reporter: Ryan Ernst
Assignee: Ryan Ernst
 Fix For: 4.9, 5.0

 Attachments: SOLR-6017.patch


 The SimpleQParser uses getAnalyzer(), but it should be getQueryAnalyzer().



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5495) Recovery strategy for leader partitioned from replica case.

2014-04-25 Thread Mark Miller (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981762#comment-13981762
 ] 

Mark Miller commented on SOLR-5495:
---

Awesome! Hope to read that closely this weekend. 

 Recovery strategy for leader partitioned from replica case.
 ---

 Key: SOLR-5495
 URL: https://issues.apache.org/jira/browse/SOLR-5495
 Project: Solr
  Issue Type: Bug
Reporter: Mark Miller
Assignee: Timothy Potter
 Attachments: SOLR-5495.patch, SOLR-5495.patch


 We need to work out a strategy for the case of:
 Leader and replicas can still talk to ZooKeeper, Leader cannot talk to 
 replica.
 We punted on this in initial design, but I'd like to get something in.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6019) Managed schema file does not show up in the Files UI

2014-04-25 Thread Stefan Matheis (steffkes) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981767#comment-13981767
 ] 

Stefan Matheis (steffkes) commented on SOLR-6019:
-

I vaguely remember another issue, which discussed that. not sure about the 
reasons that were mentioned - i recall it had something to do with confusing 
the user (or rather, try to avoid that) so in case of using managed schema, it 
does not have a schema.xml.

 Managed schema file does not show up in the Files UI
 --

 Key: SOLR-6019
 URL: https://issues.apache.org/jira/browse/SOLR-6019
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis, web gui
Affects Versions: 4.7.2, 4.8
Reporter: Shalin Shekhar Mangar
 Attachments: 6019-missing-managed-schema.png


 When running with the schema-less example, I noticed that the managed-schema 
 file does not show in the Files section of the Admin UI. This can be 
 confusing for a user. To make sure it was not a caching issue on the browser, 
 I closed and opened the UI again in a new tab. I also restarted Solr and 
 still the managed-schema is not visible in the Files section. Interestingly, 
 the schema.xml.bak does show up. A screenshot of the UI is attached.
 It is possible that this bug affects other managed resources as well such as 
 synonyms but I haven't tested that yet.
 The schema browser works fine though.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-6019) Managed schema file does not show up in the Files UI

2014-04-25 Thread Stefan Matheis (steffkes) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981770#comment-13981770
 ] 

Stefan Matheis (steffkes) commented on SOLR-6019:
-

found one comment in 
[ShowFileRequestHandler.java|http://svn.apache.org/viewvc/lucene/dev/trunk/solr/core/src/java/org/apache/solr/handler/admin/ShowFileRequestHandler.java?view=markup#l278]
 on line 279ff, leading towards SOLR-5455

 Managed schema file does not show up in the Files UI
 --

 Key: SOLR-6019
 URL: https://issues.apache.org/jira/browse/SOLR-6019
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis, web gui
Affects Versions: 4.7.2, 4.8
Reporter: Shalin Shekhar Mangar
 Attachments: 6019-missing-managed-schema.png


 When running with the schema-less example, I noticed that the managed-schema 
 file does not show in the Files section of the Admin UI. This can be 
 confusing for a user. To make sure it was not a caching issue on the browser, 
 I closed and opened the UI again in a new tab. I also restarted Solr and 
 still the managed-schema is not visible in the Files section. Interestingly, 
 the schema.xml.bak does show up. A screenshot of the UI is attached.
 It is possible that this bug affects other managed resources as well such as 
 synonyms but I haven't tested that yet.
 The schema browser works fine though.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20-ea-b05) - Build # 10161 - Failure!

2014-04-25 Thread Policeman Jenkins Server
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10161/
Java: 32bit/jdk1.8.0_20-ea-b05 -client -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  
org.apache.lucene.analysis.icu.TestICUNormalizer2CharFilter.testRandomStrings

Error Message:
startOffset 824 expected:5397 but was:5398

Stack Trace:
java.lang.AssertionError: startOffset 824 expected:5397 but was:5398
at 
__randomizedtesting.SeedInfo.seed([FF2E1D9492AE57B7:77A71D2A31AA0082]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:181)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:294)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:298)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:858)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:613)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:511)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:435)
at 
org.apache.lucene.analysis.icu.TestICUNormalizer2CharFilter.testRandomStrings(TestICUNormalizer2CharFilter.java:189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 

[jira] [Updated] (SOLR-445) Update Handlers abort with bad documents

2014-04-25 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/SOLR-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe updated SOLR-445:
---

Attachment: SOLR-445-alternative.patch

My simple test to use with SolrCloud fails (not 100% of the times, but very 
frequently). This is my understanding of the problem:
It works only in the case of the update arriving to the shard leader (as it 
would fail while adding the doc locally), but if the update needs to be 
forwarded to the leader, then it will not work. 
If the request is forwarded to the leader it is done asynchronically and the 
DistributedUpdateProcessor tracks the errors internally. Finally, after all the 
docs where processed  the “finish” method is called and the 
DistributedUpdateProcessor will add one of the exceptions to the response. This 
is a problem because “processAdd” never really fails as the 
TolerantUpdateProcessor is expecting. It also can’t know the total number of 
errors, this is counted internally in the DistributedUpdateProcessor. 

As a side note, this DistributedUpdateProcessor behavior makes it “tolerant”, 
but only in some cases? A request like this:

addinvalid-doc/add
addvalid-doc/add
addvalid-doc/add


would leave Solr in a different state depending on who is receiving the request 
(the shard leader or a replica/follower). Is this expected?


 Update Handlers abort with bad documents
 

 Key: SOLR-445
 URL: https://issues.apache.org/jira/browse/SOLR-445
 Project: Solr
  Issue Type: Improvement
  Components: update
Affects Versions: 1.3
Reporter: Will Johnson
 Fix For: 4.9, 5.0

 Attachments: SOLR-445-3_x.patch, SOLR-445-alternative.patch, 
 SOLR-445-alternative.patch, SOLR-445-alternative.patch, 
 SOLR-445-alternative.patch, SOLR-445.patch, SOLR-445.patch, SOLR-445.patch, 
 SOLR-445.patch, SOLR-445_3x.patch, solr-445.xml


 Has anyone run into the problem of handling bad documents / failures mid 
 batch.  Ie:
 add
   doc
 field name=id1/field
   /doc
   doc
 field name=id2/field
 field name=myDateFieldI_AM_A_BAD_DATE/field
   /doc
   doc
 field name=id3/field
   /doc
 /add
 Right now solr adds the first doc and then aborts.  It would seem like it 
 should either fail the entire batch or log a message/return a code and then 
 continue on to add doc 3.  Option 1 would seem to be much harder to 
 accomplish and possibly require more memory while Option 2 would require more 
 information to come back from the API.  I'm about to dig into this but I 
 thought I'd ask to see if anyone had any suggestions, thoughts or comments.   
  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: VOTE: RC1 Release apache-solr-ref-guide-4.8.pdf

2014-04-25 Thread Steve Rowe
+1

Steve



On Fri, Apr 25, 2014 at 5:38 PM, Chris Hostetter
hossman_luc...@fucit.orgwrote:


 (Note: cross posted to general, please confine replies to dev@lucene)

 Please VOTE to release the following RC1 as apache-solr-ref-guide-4.8.pdf
 ...

 https://dist.apache.org/repos/dist/dev/lucene/solr/ref-
 guide/apache-solr-ref-guide-4.8-RC1


 The notes I previously mentioned regarding RC0 apply to this RC as well...

 1) Due to a known bug in confluence, the PDFs it generates are much bigger
 then they should be.  This bug has been fixed in the latest version of
 confluence, but cwiki.apache.rog has not yet been updated.  For that
 reason, I have manually run a small tool against the PDF to fix the size
 (see SOLR-5819).  The first time i tried this approach, it inadvertantly
 removed the Index (aka: Table of Contents, or Bookmarks depending on what
 PDF reader client you use).  I've already fixed this, but if you notice
 anything else unusual about this PDF compared to previous versions please
 speak up so we can see if it's a result of this post-processing and try to
 fix it.

 2) This is the first ref guide release where we've started using a special
 confluence macro for any lucene/solr javadoc links.  The up side is that
 all javadoc links in this 4.8 ref guide will now correctly point to the 4.8
 javadocs on lucene.apache.org -- the down side is that this means none of
 those links currently work, since the 4.8 code release is still ongoing and
 the website has not yet been updated.

 Because of #2, I intend to leave this ref guide vote open until the 4.8
 code release is final - that way we won't officially be releasing this doc
 until the 4.8 javadocs are uploaded and all the links work properly.



 -Hoss
 http://www.lucidworks.com/



[jira] [Commented] (SOLR-5495) Recovery strategy for leader partitioned from replica case.

2014-04-25 Thread Otis Gospodnetic (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981852#comment-13981852
 ] 

Otis Gospodnetic commented on SOLR-5495:


bq. One idea I had was to use JMX to raise some event / notification to allow 
monitoring tools to alert ops team of this issue. Curious if there's anything 
else in SolrCloud related to notifying of issues that need ops attention?

+1 for bringing this up - not that I'm aware of, but I also didn't look very 
closely.  This may deserve a standalone JIRA!  Perhaps relying on ZK for 
notifications would work?


 Recovery strategy for leader partitioned from replica case.
 ---

 Key: SOLR-5495
 URL: https://issues.apache.org/jira/browse/SOLR-5495
 Project: Solr
  Issue Type: Bug
Reporter: Mark Miller
Assignee: Timothy Potter
 Attachments: SOLR-5495.patch, SOLR-5495.patch


 We need to work out a strategy for the case of:
 Leader and replicas can still talk to ZooKeeper, Leader cannot talk to 
 replica.
 We punted on this in initial design, but I'd like to get something in.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: maxThreads in Jetty

2014-04-25 Thread Otis Gospodnetic
I think moving away from Jetty and going to Netty or something like that is
on the radar for 5, no?

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr  Elasticsearch Support * http://sematext.com/


On Fri, Apr 25, 2014 at 8:10 AM, Toke Eskildsen t...@statsbiblioteket.dkwrote:

 On Fri, 2014-04-25 at 12:55 +0200, Mark Miller wrote:
  Too avoid distributed deadlock.

 Ah. I did not consider that.

  It would be better to add support for setting an internal port and
  using two thread pools - before 5x though, that would be a fairly
  unfriendly out of the box solution for non jetty containers.
 
 It is very simple in tomcat to add another connector with a given
 maxThreads, but of course it still involves editing a file that is
 otherwise not touched. I can see why that might be a problem. I do not
 know how difficult it is in other Web containers.

 Do you know if a limitation on threads is on the radar for Solr 5?

 Thank you,
 Toke Eskildsen, State and Univeristy Library, Denmark




 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




[jira] [Commented] (SOLR-6019) Managed schema file does not show up in the Files UI

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981874#comment-13981874
 ] 

Shalin Shekhar Mangar commented on SOLR-6019:
-

Thanks Stefan. I'd still consider this a bug. We should be able to read the 
file even if it cannot be edited. Also the editing feature itself is limited to 
trunk.

 Managed schema file does not show up in the Files UI
 --

 Key: SOLR-6019
 URL: https://issues.apache.org/jira/browse/SOLR-6019
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis, web gui
Affects Versions: 4.7.2, 4.8
Reporter: Shalin Shekhar Mangar
 Attachments: 6019-missing-managed-schema.png


 When running with the schema-less example, I noticed that the managed-schema 
 file does not show in the Files section of the Admin UI. This can be 
 confusing for a user. To make sure it was not a caching issue on the browser, 
 I closed and opened the UI again in a new tab. I also restarted Solr and 
 still the managed-schema is not visible in the Files section. Interestingly, 
 the schema.xml.bak does show up. A screenshot of the UI is attached.
 It is possible that this bug affects other managed resources as well such as 
 synonyms but I haven't tested that yet.
 The schema browser works fine though.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-5973) Pluggable Ranking Collectors

2014-04-25 Thread Joel Bernstein (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-5973:
-

Attachment: SOLR-5973.patch

Added mergeStrategy test using sort criteria other then score.

 Pluggable Ranking Collectors
 

 Key: SOLR-5973
 URL: https://issues.apache.org/jira/browse/SOLR-5973
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Joel Bernstein
Assignee: Joel Bernstein
Priority: Minor
 Fix For: 4.9

 Attachments: SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, 
 SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, SOLR-5973.patch, 
 SOLR-5973.patch, SOLR-5973.patch


 This ticket adds the ability to plugin a custom ranking collector to Solr. 
 The proposed design is much simpler then SOLR-4465, which includes 
 configuration support and support for pluggable analytics collectors.
 In this design, a CollectorFactory can be set onto the ResponseBuilder by a 
 custom SearchComponent. The CollectorFactory is then used to inject a custom 
 TopDocsCollector into the SolrIndexSearcher.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)

2014-04-25 Thread Shawn Heisey (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981880#comment-13981880
 ] 

Shawn Heisey commented on SOLR-5981:


[~jdyer], I'm not very familiar with DIH code.  Are you saying that I shouldn't 
commit this patch?


 Please change method visibility of getSolrWriter in DataImportHandler to 
 public (or at least protected)
 ---

 Key: SOLR-5981
 URL: https://issues.apache.org/jira/browse/SOLR-5981
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.0
 Environment: Linux 3.13.9-200.fc20.x86_64
 Solr 4.6.0
Reporter: Aaron LaBella
Assignee: Shawn Heisey
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-5981.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 I've been using the org.apache.solr.handler.dataimport.DataImportHandler for 
 a bit and it's an excellent model and architecture.  I'd like to extend the 
 usage of it to plugin my own DIHWriter, but, the code doesn't allow for it.  
 Please change ~line 227 in the DataImportHander class to be:
 public SolrWriter getSolrWriter
 instead of:
 private SolrWriter getSolrWriter
 or, at a minimum, protected, so that I can extend DataImportHandler and 
 override this method.
 Thank you *sincerely* in advance for the quick turn-around on this.  If the 
 change can be made in 4.6.0 and upstream, that'd be ideal.
 Thanks!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-5981) Please change method visibility of getSolrWriter in DataImportHandler to public (or at least protected)

2014-04-25 Thread Shalin Shekhar Mangar (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981887#comment-13981887
 ] 

Shalin Shekhar Mangar commented on SOLR-5981:
-

bq. I did a proof of concept to use the DataImportHandler framework to import 
into mongodb

Oh, that's awesome. I wanted to do something similar in SOLR-853 but didn't get 
the time to work on it. I'd like to have DIH as an API which can be used e.g. 
with SolrJ.

 Please change method visibility of getSolrWriter in DataImportHandler to 
 public (or at least protected)
 ---

 Key: SOLR-5981
 URL: https://issues.apache.org/jira/browse/SOLR-5981
 Project: Solr
  Issue Type: Improvement
  Components: contrib - DataImportHandler
Affects Versions: 4.0
 Environment: Linux 3.13.9-200.fc20.x86_64
 Solr 4.6.0
Reporter: Aaron LaBella
Assignee: Shawn Heisey
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-5981.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 I've been using the org.apache.solr.handler.dataimport.DataImportHandler for 
 a bit and it's an excellent model and architecture.  I'd like to extend the 
 usage of it to plugin my own DIHWriter, but, the code doesn't allow for it.  
 Please change ~line 227 in the DataImportHander class to be:
 public SolrWriter getSolrWriter
 instead of:
 private SolrWriter getSolrWriter
 or, at a minimum, protected, so that I can extend DataImportHandler and 
 override this method.
 Thank you *sincerely* in advance for the quick turn-around on this.  If the 
 change can be made in 4.6.0 and upstream, that'd be ideal.
 Thanks!



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org