date:20140605

2014-06-05 Thread ASF subversion and git services (JIRA)

Michael McCandless created LUCENE-5737:
--

 Summary: TestControlledRTReopenThread.testCRTReopen relies on wall 
clock time
 Key: LUCENE-5737
 URL: https://issues.apache.org/jira/browse/LUCENE-5737
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Michael McCandless


I'm disabling this test for now; I think to fix it we need to be able to make 
the timer pluggable in this class, and mock a more controllable wall clock 
for the test.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5737) TestControlledRTReopenThread.testCRTReopen relies on wall clock time


[ 
https://issues.apache.org/jira/browse/LUCENE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018618#comment-14018618
 ] 

ASF subversion and git services commented on LUCENE-5737:
-

Commit 1600573 from [~mikemccand] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1600573 ]

LUCENE-5737: disable this test for now

 TestControlledRTReopenThread.testCRTReopen relies on wall clock time
 

 Key: LUCENE-5737
 URL: https://issues.apache.org/jira/browse/LUCENE-5737
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Michael McCandless

 I'm disabling this test for now; I think to fix it we need to be able to make 
 the timer pluggable in this class, and mock a more controllable wall clock 
 for the test.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5737) TestControlledRTReopenThread.testCRTReopen relies on wall clock time

2014-06-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018619#comment-14018619
 ] 

ASF subversion and git services commented on LUCENE-5737:
-

Commit 1600575 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1600575 ]

LUCENE-5737: disable this test for now

 TestControlledRTReopenThread.testCRTReopen relies on wall clock time
 

 Key: LUCENE-5737
 URL: https://issues.apache.org/jira/browse/LUCENE-5737
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Michael McCandless

 I'm disabling this test for now; I think to fix it we need to be able to make 
 the timer pluggable in this class, and mock a more controllable wall clock 
 for the test.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers

2014-06-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018656#comment-14018656
 ] 

Adrien Grand commented on LUCENE-5703:
--

{quote}
   * System.arraycopy
   * big fat warnings on these that they are unsafe (they are not part of the 
official index format, so maybe thats ok).
   * they could keep ahold of their file descriptors and override merge() to 
stream the data from the file.
{quote}

I'm hesitating between the 1st and 2nd idea. I'm not a fan of the 3rd idea as 
it would make merging a bit more complex although I really like how simple it 
is today.

Since the default codec doesn't have the issue and since this issue already 
exists in other index formats (eg. the payload of DirectPostingsFormat), I 
think it would be fine to just have a warning?

 Don't allocate/copy bytes all the time in binary DV producers
 -

 Key: LUCENE-5703
 URL: https://issues.apache.org/jira/browse/LUCENE-5703
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, 
 LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch


 Our binary doc values producers keep on creating new {{byte[]}} arrays and 
 copying bytes when a value is requested, which likely doesn't help 
 performance. This has been done because of the way fieldcache consumers used 
 the API, but we should try to fix it in 5.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-SmokeRelease-4.8 - Build # 6 - Still Failing

2014-06-05 Thread Robert Muir

i committed a fix.

On Thu, Jun 5, 2014 at 5:17 AM, Apache Jenkins Server
jenk...@builds.apache.org wrote:
 Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.8/6/

 No tests ran.

 Build Log:
 [...truncated 52791 lines...]
 prepare-release-no-sign:
 [mkdir] Created dir: 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/lucene/build/fakeRelease
  [copy] Copying 431 files to 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/lucene/build/fakeRelease/lucene
  [copy] Copying 239 files to 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/lucene/build/fakeRelease/solr
  [exec] JAVA7_HOME is /home/hudson/tools/java/latest1.7
  [exec] NOTE: output encoding is US-ASCII
  [exec]
  [exec] Load release URL 
 file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/lucene/build/fakeRelease/...
  [exec]
  [exec] Test Lucene...
  [exec]   test basics...
  [exec]   get KEYS
  [exec] 0.1 MB in 0.01 sec (20.4 MB/sec)
  [exec]   check changes HTML...
  [exec]   download lucene-4.8.0-src.tgz...
  [exec] 27.5 MB in 0.04 sec (670.9 MB/sec)
  [exec] verify md5/sha1 digests
  [exec]   download lucene-4.8.0.tgz...
  [exec] 61.3 MB in 0.09 sec (654.7 MB/sec)
  [exec] verify md5/sha1 digests
  [exec]   download lucene-4.8.0.zip...
  [exec] 70.9 MB in 0.14 sec (511.3 MB/sec)
  [exec] verify md5/sha1 digests
  [exec]   unpack lucene-4.8.0.tgz...
  [exec] verify JAR metadata/identity/no javax.* or java.* classes...
  [exec] test demo with 1.7...
  [exec]   got 5687 hits for query lucene
  [exec] check Lucene's javadoc JAR
  [exec]   unpack lucene-4.8.0.zip...
  [exec] verify JAR metadata/identity/no javax.* or java.* classes...
  [exec] test demo with 1.7...
  [exec]   got 5687 hits for query lucene
  [exec] check Lucene's javadoc JAR
  [exec]   unpack lucene-4.8.0-src.tgz...
  [exec] make sure no JARs/WARs in src dist...
  [exec] run ant validate
  [exec] run tests w/ Java 7 and 
 testArgs='-Dtests.jettyConnector=Socket  -Dtests.disableHdfs=true'...
  [exec] test demo with 1.7...
  [exec]   got 249 hits for query lucene
  [exec] generate javadocs w/ Java 7...
  [exec]
  [exec] Crawl/parse...
  [exec]
  [exec] Verify...
  [exec]
  [exec] Test Solr...
  [exec]   test basics...
  [exec]   get KEYS
  [exec] 0.1 MB in 0.00 sec (101.5 MB/sec)
  [exec]   check changes HTML...
  [exec] Traceback (most recent call last):
  [exec]   File 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/dev-tools/scripts/smokeTestRelease.py,
  line 1347, in module
  [exec] main()
  [exec]   File 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/dev-tools/scripts/smokeTestRelease.py,
  line 1291, in main
  [exec] smokeTest(baseURL, svnRevision, version, tmpDir, isSigned, 
 testArgs)
  [exec]   File 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/dev-tools/scripts/smokeTestRelease.py,
  line 1333, in smokeTest
  [exec] checkSigs('solr', solrPath, version, tmpDir, isSigned)
  [exec]   File 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/dev-tools/scripts/smokeTestRelease.py,
  line 410, in checkSigs
  [exec] testChanges(project, version, changesURL)
  [exec]   File 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/dev-tools/scripts/smokeTestRelease.py,
  line 458, in testChanges
  [exec] checkChangesContent(s, version, changesURL, project, True)
  [exec]   File 
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/dev-tools/scripts/smokeTestRelease.py,
  line 485, in checkChangesContent
  [exec] raise RuntimeError('incorrect issue (_ instead of -) in %s: 
 %s' % (name, m.group(1)))
  [exec] RuntimeError: incorrect issue (_ instead of -) in 
 file:///usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/lucene/build/fakeRelease/solr/changes/Changes.html:
  SOLR_6029

 BUILD FAILED
 /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-SmokeRelease-4.8/build.xml:387:
  exec returned: 1

 Total time: 54 minutes 2 seconds
 Build step 'Invoke Ant' marked build as failure
 Email was triggered for: Failure
 Sending email for trigger: Failure




 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2010 - Still Failing

2014-06-05 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2010/

All tests passed

Build Log:
[...truncated 29357 lines...]
check-licenses:
 [echo] License check under: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/log4j-1.2.16.jar
 [licenses] EXPECTED sha1 checksum file : 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/log4j-1.2.16.jar.sha1

[...truncated 1 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:70:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62:
 License check failed. Check the logs.

Total time: 129 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5716) Track file handle leaks (FileDescriptor, NIO Path SPI and Socket mostly).


[ 
https://issues.apache.org/jira/browse/LUCENE-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018701#comment-14018701
 ] 

ASF subversion and git services commented on LUCENE-5716:
-

Commit 1600614 from [~dawidweiss] in branch 'dev/branches/LUCENE-5716'
[ https://svn.apache.org/r1600614 ]

Branch for LUCENE-5716

 Track file handle leaks (FileDescriptor, NIO Path SPI and Socket mostly).
 -

 Key: LUCENE-5716
 URL: https://issues.apache.org/jira/browse/LUCENE-5716
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor





--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6119) TestReplicationHandler attempts to remove open folders

2014-06-05 Thread Varun Thacker (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018715#comment-14018715
 ] 

Varun Thacker commented on SOLR-6119:
-

Dawid, can we resolve this issue?

 TestReplicationHandler attempts to remove open folders
 --

 Key: SOLR-6119
 URL: https://issues.apache.org/jira/browse/SOLR-6119
 Project: Solr
  Issue Type: Bug
Reporter: Dawid Weiss
Priority: Minor
 Attachments: SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch, 
 SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch


 TestReplicationHandler has a weird logic around the 'snapDir' variable. It 
 attempts to remove snapshot folders, even though they're not closed yet. My 
 recent patch uncovered the bug but I don't know how to fix it cleanly -- the 
 test itself seems to be very fragile (for example I don't understand the 
 'namedBackup' variable which is always set to true, yet there are 
 conditionals around it).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6119) TestReplicationHandler attempts to remove open folders

2014-06-05 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018725#comment-14018725
 ] 

Dawid Weiss commented on SOLR-6119:
---

I think so, thanks!

 TestReplicationHandler attempts to remove open folders
 --

 Key: SOLR-6119
 URL: https://issues.apache.org/jira/browse/SOLR-6119
 Project: Solr
  Issue Type: Bug
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch, 
 SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch


 TestReplicationHandler has a weird logic around the 'snapDir' variable. It 
 attempts to remove snapshot folders, even though they're not closed yet. My 
 recent patch uncovered the bug but I don't know how to fix it cleanly -- the 
 test itself seems to be very fragile (for example I don't understand the 
 'namedBackup' variable which is always set to true, yet there are 
 conditionals around it).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-6119) TestReplicationHandler attempts to remove open folders

2014-06-05 Thread Dawid Weiss (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss resolved SOLR-6119.
---

   Resolution: Fixed
Fix Version/s: 5.0
   4.9
 Assignee: Dawid Weiss

 TestReplicationHandler attempts to remove open folders
 --

 Key: SOLR-6119
 URL: https://issues.apache.org/jira/browse/SOLR-6119
 Project: Solr
  Issue Type: Bug
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch, 
 SOLR-6119.patch, SOLR-6119.patch, SOLR-6119.patch


 TestReplicationHandler has a weird logic around the 'snapDir' variable. It 
 attempts to remove snapshot folders, even though they're not closed yet. My 
 recent patch uncovered the bug but I don't know how to fix it cleanly -- the 
 test itself seems to be very fragile (for example I don't understand the 
 'namedBackup' variable which is always set to true, yet there are 
 conditionals around it).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5733) Minor PackedInts API cleanups

2014-06-05 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-5733.
--

Resolution: Fixed

 Minor PackedInts API cleanups
 -

 Key: LUCENE-5733
 URL: https://issues.apache.org/jira/browse/LUCENE-5733
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Trivial
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5733.patch


 The PackedInts API has quite some history now and some of its methods are not 
 used anymore, eg. PackedInts.Reader.hasArray. I'd like to remove them.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5733) Minor PackedInts API cleanups


[ 
https://issues.apache.org/jira/browse/LUCENE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018743#comment-14018743
 ] 

ASF subversion and git services commented on LUCENE-5733:
-

Commit 1600637 from [~jpountz] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1600637 ]

LUCENE-5733: Remove PackedInts.Reader.(has|get)Array and move getBitsPerValue 
to PackedInts.Mutable.

 Minor PackedInts API cleanups
 -

 Key: LUCENE-5733
 URL: https://issues.apache.org/jira/browse/LUCENE-5733
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Trivial
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5733.patch


 The PackedInts API has quite some history now and some of its methods are not 
 used anymore, eg. PackedInts.Reader.hasArray. I'd like to remove them.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6138) Solr core load limit

2014-06-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018751#comment-14018751
 ] 

HuangTongwen commented on SOLR-6138:


Thank you for your replys.I know that it is zookeeper's znode limit.But I 
tickout the 1MB limit.Then it happened like that.Sometime I create core with 
the parameter onStartup that I can enlarge cores more than 4000+ cores.Even if 
I use it than I increase cores.But I don't the loggers in solr board why I 
reload would happened to some errors,I think that it isn't znode limit.Any 
other error?

 Solr core load limit
 

 Key: SOLR-6138
 URL: https://issues.apache.org/jira/browse/SOLR-6138
 Project: Solr
  Issue Type: Bug
  Components: clients - java
Affects Versions: 4.6, 4.7.1
 Environment: ubuntu 12.04
 memery 20G
Reporter: HuangTongwen
  Labels: test
   Original Estimate: 840h
  Remaining Estimate: 840h

 We want to enrich our search ability by solr.We do an exercise for test that 
 how many cores in one machine solr cores can support.
 We find we can create more than 2000 cores without datas in one machine.But 
 when we create cores with data ,we just can create about 1000 cores,after 
 more t han 1000 cores,we meet many errors like following I will apend it .If 
 you have meets the same or similar problem,please tell me.
 I would be grateful if you could help me.
 Hear are some errors:
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:46:15  ERROR   ShardLeaderElectionContext  There was a problem 
 trying to register as the 
 leader:org.apache.zookeeper.KeeperException$NodeExistsException: 
 KeeperErrorCode = NodeExists for /collections/ctest.test.3521/leaders/shard1
 09:46:15  WARNElectionContext cancelElection did not find 
 election node to remove
 09:46:16  WARNRecoveryStrategyStopping recovery for 
 zkNodeName=core_node1core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyError while trying to recover. 
 core=ctest.test.3521:org.apache.solr.common.SolrException: No registered 
 leader was found, collection:ctest.test.3521 slice:shard1
 09:46:17  ERROR   RecoveryStrategyRecovery failed - trying 
 again... (0) core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyRecovery failed - interrupted. 
 core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyRecovery failed - I give up. 
 core=ctest.test.3521

[jira] [Commented] (SOLR-6129) DateFormatTransformer doesn't resolve dateTimeFormat

2014-06-05 Thread Aaron LaBella (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018763#comment-14018763
 ] 

Aaron LaBella commented on SOLR-6129:
-

Can someone have a look at this patch and hopefully commit it?  I'm looking to 
get it in for the 4.9 release.
Thanks.

 DateFormatTransformer doesn't resolve dateTimeFormat
 

 Key: SOLR-6129
 URL: https://issues.apache.org/jira/browse/SOLR-6129
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.8.1
Reporter: Aaron LaBella
 Fix For: 4.9

 Attachments: SOLR-6129.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 The DateFormatTransformer doesn't use a VariableResolver to resolve the 
 format, which could potentially be passed in via a dataimport request 
 parameter.  The attached patch is tested and working.  Please include in 4.9.
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6140) Add indicator if schema is managed and mutable to Schema REST api.

2014-06-05 Thread Christoph Strobl (JIRA)

Christoph Strobl created SOLR-6140:
--

 Summary: Add indicator if schema is managed and mutable to Schema 
REST api.
 Key: SOLR-6140
 URL: https://issues.apache.org/jira/browse/SOLR-6140
 Project: Solr
  Issue Type: Improvement
  Components: Schema and Analysis
Reporter: Christoph Strobl


The current response to {{/collection/schema}} does not include any hint if the 
schema can be modified using the schema rest API. 

It would be great to have this information available via an additional 
attribute in the response and/or as separate entry point

{{collection/schema/managed?wt=json}}

{code:javascript}
{
  responseHeader:{
status:0,
QTime:1
  },
  managed:true
}
{code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6141) Allow removing fields via Schema REST Api.

2014-06-05 Thread Christoph Strobl (JIRA)

Christoph Strobl created SOLR-6141:
--

 Summary: Allow removing fields via Schema REST Api.
 Key: SOLR-6141
 URL: https://issues.apache.org/jira/browse/SOLR-6141
 Project: Solr
  Issue Type: Improvement
  Components: Schema and Analysis
Reporter: Christoph Strobl


It would be nice if it was possible to remove fields from the schema by sending

{{curl -X DELETE /collection/schema/fieldname}}

This would affect solrj as well as the only available methods are {{GET, POST}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1622 - Failure!

2014-06-05 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1622/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.handler.TestReplicationHandlerBackup: 1) Thread[id=1179, 
name=Thread-381, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]   
  at java.net.SocketInputStream.socketRead0(Native Method) at 
java.net.SocketInputStream.read(SocketInputStream.java:150) at 
java.net.SocketInputStream.read(SocketInputStream.java:121) at 
java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at 
java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at 
java.io.BufferedInputStream.read(BufferedInputStream.java:345) at 
sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:701) at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1534)
 at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
 at java.net.URL.openStream(URL.java:1038) at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:328)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   1) Thread[id=1179, name=Thread-381, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:701)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1534)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
at java.net.URL.openStream(URL.java:1038)
at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:328)
at __randomizedtesting.SeedInfo.seed([69441328FAB793D3]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=1179, name=Thread-381, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup] at 
java.net.SocketInputStream.socketRead0(Native Method) at 
java.net.SocketInputStream.read(SocketInputStream.java:150) at 
java.net.SocketInputStream.read(SocketInputStream.java:121) at 
java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at 
java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at 
java.io.BufferedInputStream.read(BufferedInputStream.java:345) at 
sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:701) at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1534)
 at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
 at java.net.URL.openStream(URL.java:1038) at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:328)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=1179, name=Thread-381, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:701)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1534)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
at java.net.URL.openStream(URL.java:1038)
at

[jira] [Created] (SOLR-6142) Solr Core pom.xml should not depend on Jetty and others

2014-06-05 Thread Vladimir Kulev (JIRA)

Vladimir Kulev created SOLR-6142:


 Summary: Solr Core pom.xml should not depend on Jetty and others
 Key: SOLR-6142
 URL: https://issues.apache.org/jira/browse/SOLR-6142
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.8.1
Reporter: Vladimir Kulev


Test dependencies should be optional in generated pom.xml, as it was before in 
4.6.1
Current situation makes embedded Solr use very complicated.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 5?

2014-06-05 Thread Yonik Seeley

On Wed, Jun 4, 2014 at 4:14 PM, Jack Krupansky j...@basetechnology.com wrote:
 I'll offer a prediction: 5.0 will happen when the Lucene guys at
 Elasticsearch come up with some great new ideas for how to leapfrog Solr!
 (And then we watch how the Heliosearch guys respond to that!)

Well, today I'm working on native code faceting (single valued fields first...)

I'll be talking about it at Lucene/Solr Revolution provided my talk is
accepted.  You can throw up to 3 votes toward it here:
http://lucenerevolution.uservoice.com/forums/254256-internals-track/suggestions/5995635-native-code-and-off-heap-data-structures-for-solr

-Yonik
http://heliosearch.org - facet functions, subfacets, off-heap filtersfieldcache

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers


[ 
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018893#comment-14018893
 ] 

ASF subversion and git services commented on LUCENE-5703:
-

Commit 1600688 from [~rcmuir] in branch 'dev/trunk'
[ https://svn.apache.org/r1600688 ]

LUCENE-5703: BinaryDocValues producers don't allocate or copy bytes on each 
access anymore

 Don't allocate/copy bytes all the time in binary DV producers
 -

 Key: LUCENE-5703
 URL: https://issues.apache.org/jira/browse/LUCENE-5703
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, 
 LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch


 Our binary doc values producers keep on creating new {{byte[]}} arrays and 
 copying bytes when a value is requested, which likely doesn't help 
 performance. This has been done because of the way fieldcache consumers used 
 the API, but we should try to fix it in 5.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed

Simon Willnauer created LUCENE-5738:
---

 Summary: NativeLock is release if Lock is closed after obtain 
failed
 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
 Fix For: 4.9, 5.0


if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
close if if it fails another process will be able to obtain it. This is pretty 
trappy though. If you execute the main class twice the problem becomes pretty 
obvious.

{noformat}
import org.apache.lucene.store.Lock;
import org.apache.lucene.store.NativeFSLockFactory;

import java.io.File;
import java.io.IOException;

public class TestLock {
 public static void main(String[] foo) throws IOException, InterruptedException 
{
NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
File(/tmp));
Lock lock = lockFactory.makeLock(LOCK);
if (lock.obtain()) {
System.out.println(OBTAINED);
} else {
lock.close();
System.out.println(FAILED);
}
if (lock.obtain()) {
System.out.println(OBTAINED AGAIN);
} else {
lock.close();
System.out.println(FAILED on Second);
}
Thread.sleep(Integer.MAX_VALUE);
}
}

{noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed


 [ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-5738:


Description: 
if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
close if if it fails another process will be able to obtain it. This is pretty 
trappy though. If you execute the main class twice the problem becomes pretty 
obvious.

{noformat}
import org.apache.lucene.store.Lock;
import org.apache.lucene.store.NativeFSLockFactory;

import java.io.File;
import java.io.IOException;

public class TestLock {
 public static void main(String[] foo) throws IOException, InterruptedException 
{
NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
File(/tmp));
Lock lock = lockFactory.makeLock(LOCK);
if (lock.obtain()) {
System.out.println(OBTAINED);
} else {
lock.close();
System.out.println(FAILED);
}
// try it again and close it if it fails
lock = lockFactory.makeLock(LOCK);
if (lock.obtain()) {
System.out.println(OBTAINED AGAIN);
} else {
lock.close();
System.out.println(FAILED on Second);
}
Thread.sleep(Integer.MAX_VALUE);
}
}

{noformat}

  was:
if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
close if if it fails another process will be able to obtain it. This is pretty 
trappy though. If you execute the main class twice the problem becomes pretty 
obvious.

{noformat}
import org.apache.lucene.store.Lock;
import org.apache.lucene.store.NativeFSLockFactory;

import java.io.File;
import java.io.IOException;

public class TestLock {
 public static void main(String[] foo) throws IOException, InterruptedException 
{
NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
File(/tmp));
Lock lock = lockFactory.makeLock(LOCK);
if (lock.obtain()) {
System.out.println(OBTAINED);
} else {
lock.close();
System.out.println(FAILED);
}
if (lock.obtain()) {
System.out.println(OBTAINED AGAIN);
} else {
lock.close();
System.out.println(FAILED on Second);
}
Thread.sleep(Integer.MAX_VALUE);
}
}

{noformat}


 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close();
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed

2014-06-05 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-5738:


Description: 
if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
close if if it fails another process will be able to obtain it. This is pretty 
trappy though. If you execute the main class twice the problem becomes pretty 
obvious.

{noformat}
import org.apache.lucene.store.Lock;
import org.apache.lucene.store.NativeFSLockFactory;

import java.io.File;
import java.io.IOException;

public class TestLock {
 public static void main(String[] foo) throws IOException, InterruptedException 
{
NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
File(/tmp));
Lock lock = lockFactory.makeLock(LOCK);
if (lock.obtain()) {
System.out.println(OBTAINED);
} else {
lock.close();
System.out.println(FAILED);
}
// try it again and close it if it fails
lock = lockFactory.makeLock(LOCK); //  this is a new lock
if (lock.obtain()) {
System.out.println(OBTAINED AGAIN);
} else {
lock.close(); //  this releases the lock we obtained
System.out.println(FAILED on Second);
}
Thread.sleep(Integer.MAX_VALUE);
}
}

{noformat}

  was:
if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
close if if it fails another process will be able to obtain it. This is pretty 
trappy though. If you execute the main class twice the problem becomes pretty 
obvious.

{noformat}
import org.apache.lucene.store.Lock;
import org.apache.lucene.store.NativeFSLockFactory;

import java.io.File;
import java.io.IOException;

public class TestLock {
 public static void main(String[] foo) throws IOException, InterruptedException 
{
NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
File(/tmp));
Lock lock = lockFactory.makeLock(LOCK);
if (lock.obtain()) {
System.out.println(OBTAINED);
} else {
lock.close();
System.out.println(FAILED);
}
// try it again and close it if it fails
lock = lockFactory.makeLock(LOCK);
if (lock.obtain()) {
System.out.println(OBTAINED AGAIN);
} else {
lock.close();
System.out.println(FAILED on Second);
}
Thread.sleep(Integer.MAX_VALUE);
}
}

{noformat}


 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5721) Monotonic packed could maybe be faster

[
https://issues.apache.org/jira/browse/LUCENE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018923#comment-14018923
]

ASF subversion and git services commented on LUCENE-5721:
-

Commit 1600694 from [~jpountz] in branch 'dev/trunk'
[ https://svn.apache.org/r1600694 ]

LUCENE-5721: Monotonic compression doesn't use zig-zag encoding anymore.

Monotonic packed could maybe be faster
--

Key: LUCENE-5721
URL: https://issues.apache.org/jira/browse/LUCENE-5721
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
Assignee: Adrien Grand
Attachments: LUCENE-5703.patch

This compression is used in lucene for monotonically increasing offsets, e.g.
stored fields index, dv BINARY/SORTED_SET offsets, OrdinalMap (used for
merging and faceting dv) and so on.
Today this stores a +/- deviation from an expected line of y=mx + b, where b
is the minValue for the block and m is the average delta from the previous
value. Because it can be negative, we have to do some additional work to
zigzag-decode.
Can we just instead waste a bit for every value explicitly (lower the
minValue by the min delta) so that deltas are always positive and we can have
a simpler decode? Maybe If we do this, the new guy should assert that values
are actually monotic at write-time. The current one supports mostly monotic
but do we really need that flexibility anywhere? If so it could always be
kept...

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 5?

2014-06-05 Thread Shawn Heisey

 The 4.x branch seems to be doing well enough, both from a stability
 perspective and momentum with new features.

I agree with this assessment. It is currently very easy to backport almost
all new functionality from trunk (5x) to the 4x branch. Although there are
some features unique to trunk, I don't think the list is extensive yet.

I thought that the switch to Java 7 in trunk would force a relatively
short time line to 5.0, but the Java 7 requirement itself was backported
to 4x, so that influence is no longer there.

 I'll offer a prediction: 5.0 will happen when the Lucene guys at
 Elasticsearch come up with some great new ideas for how to leapfrog Solr!
 (And then we watch how the Heliosearch guys respond to that!)

I still think that a serious push for 5.0 will not begin until backporting
code changes from trunk to 4x requires signing manual intervention


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 5?

2014-06-05 Thread Shawn Heisey

 The 4.x branch seems to be doing well enough, both from a stability
 perspective and momentum with new features.

I agree with this assessment. It is currently very easy to backport almost
all new functionality from trunk (5x) to the 4x branch. Although there are
some features unique to trunk, I don't think the list is extensive yet.

I thought that the switch to Java 7 in trunk would force a relatively
short time line to 5.0, but the Java 7 requirement itself was backported
to 4x, so that influence is no longer there.

 I'll offer a prediction: 5.0 will happen when the Lucene guys at
 Elasticsearch come up with some great new ideas for how to leapfrog Solr!
 (And then we watch how the Heliosearch guys respond to that!)

I still think that a serious push for 5.0 will not begin until backporting
small and medium sized code changes from trunk to 4x requires significant
manual intervention.

If the leapfrog situation happens as you describe, and the resulting Solr
changes can easily be backported to 4x, then I don't think it will spur a
new major release.

Thanks,
Shawn




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 5?

2014-06-05 Thread Alexandre Rafalovitch

I think Solr 5 should be a modularized deploy with plugins and plugin
registry. Either that or we will have to steal the Elephant logo from
the sister project and have to setup torrent-based software
distribution.

And I really wish HUE was a Java-based project. But, alas,

Regards,
  Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency


On Thu, Jun 5, 2014 at 11:35 PM, Shawn Heisey s...@elyograg.org wrote:
 The 4.x branch seems to be doing well enough, both from a stability
 perspective and momentum with new features.

 I agree with this assessment. It is currently very easy to backport almost
 all new functionality from trunk (5x) to the 4x branch. Although there are
 some features unique to trunk, I don't think the list is extensive yet.

 I thought that the switch to Java 7 in trunk would force a relatively
 short time line to 5.0, but the Java 7 requirement itself was backported
 to 4x, so that influence is no longer there.

 I'll offer a prediction: 5.0 will happen when the Lucene guys at
 Elasticsearch come up with some great new ideas for how to leapfrog Solr!
 (And then we watch how the Heliosearch guys respond to that!)

 I still think that a serious push for 5.0 will not begin until backporting
 code changes from trunk to 4x requires signing manual intervention


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5715) Upgrade direct dependencies known to be older than transitive dependencies


 [ 
https://issues.apache.org/jira/browse/LUCENE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated LUCENE-5715:
---

Fix Version/s: 5.0

 Upgrade direct dependencies known to be older than transitive dependencies
 --

 Key: LUCENE-5715
 URL: https://issues.apache.org/jira/browse/LUCENE-5715
 Project: Lucene - Core
  Issue Type: Task
  Components: general/build
Reporter: Steve Rowe
Assignee: Steve Rowe
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5715.patch


 LUCENE-5442 added functionality to the {{check-lib-versions}} ant task to 
 fail the build if a direct dependency's version conflicts with that of a 
 transitive dependency.
 {{ivy-ignore-conflicts.properties}} contains a list of 19 transitive 
 dependencies with versions that are newer than direct dependencies' versions: 
 https://issues.apache.org/jira/browse/LUCENE-5442?focusedCommentId=14012220page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14012220
 We should try to keep that list small.  It's likely that upgrading most of 
 those dependencies will require little effort.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Trappy behavior with default search field

2014-06-05 Thread Erick Erickson

We've all been bitten by the trappy problem with removing the text field
from schema.xml and then Solr failing to start b/c various handlers
specifically call it out.

Problem is that as people build out (particularly) SolrCloud clusters, this
innocent-seeming action is getting harder and harder to track down.

Is it worth a JIRA to address? And any clues how to address it? I started
thinking about in a _very_ superficial manner and I suspect that this is
one of those things that _seems_ easy but turns into a sticky wicket.

If we make sure and return a field that _is_ defined, then the problem
becomes even harder to detect. I mean you don't even get any warning but
don't find your docs b/c the default field isn't there and you're searching
on a different field than you think. At least the current behavior
sometimes causes Solr to not start at all.

Im not even sure it's worth doing, we currently both print an error in the
log and return an error message for a search, but wanted to gather other's
thoughts.

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018948#comment-14018948
 ] 

Simon Willnauer commented on LUCENE-5738:
-

Ok this gets more funky... I modified the test since it is not exactly what I 
was seeing
{code}
 public static void main(String[] foo) throws IOException, InterruptedException 
{
NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
File(/tmp));
Lock lock = lockFactory.makeLock(LOCK);
if (lock.obtain()) {
System.out.println(OBTAINED);
} else {
System.out.println(FAILED);
}

lock = lockFactory.makeLock(LOCK);   //  this is a new lock
// here when we call obtain we run into an OverlappingFileLockException
// this exception closes the file channel and that causes the
// internal file lock table to be invalidated. This essentially 
releases the W lock of the first descriptor
if (lock.obtain()) {
System.out.println(OBTAINED AGAIN);
} else {
System.out.println(FAILED on Second);
}
Thread.sleep(Integer.MAX_VALUE);
}
{code}

it seems that the FileChannel release all locks if it is closed.. There is some 
funky code in FileChannelImpl.java I would't be suprised if it has bugs

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6143) Bad facet counts from CollapsingQParserPlugin

2014-06-05 Thread David Fennessey (JIRA)

David Fennessey created SOLR-6143:
-

 Summary: Bad facet counts from CollapsingQParserPlugin 
 Key: SOLR-6143
 URL: https://issues.apache.org/jira/browse/SOLR-6143
 Project: Solr
  Issue Type: Bug
  Components: query parsers
Affects Versions: 4.8.1
 Environment: UNIX
Tomcat 7.0.33
SOLR 4.8.1
16 GB RAM

Reporter: David Fennessey


I'm noticing a very weird bug using the CollapsingQParserPlugin. We tried to 
use this plugin when we realized that faceting on the groups would take a 
ridiculous amount of time. To its credit, it works very quickly, however the 
facet counts that it gives are incorrect. 

We have a smallish index of about 200k documents with about with about 50k 
distinct groups within it. 

When we use the group implementation 
(group=truegroup.field=PrSKUgroup.facet=true) which I believe this attempts 
to emulate, the facet counts are totally correct. 

When we use the field collapsing implementation, it will show an incorrect 
count for the non-filtered query, but when we go to the filtered query, the 
facet count corrects itself and matches the document count. 

Here are some SOLR responses:

solrslave01:8983/index/select?q=classIDs:12fl=PrSKUfq={!collapse%20field=PrSKU}facet=truefacet.field=at_12_wood_tone

The facet field will return 

int name=Dark Wood867/int
int name=Medium Wood441/int
int name=Light Wood253/int

When I actually apply a filter query like so:

solrslave01:8983/index/select?q=classIDs:12fl=PrSKUfq={!collapse%20field=PrSKU}facet=truefacet.field=at_12_wood_tonefq=at_12_wood_tone:%22Light%20Wood%22

I actually pull back 270 results and the facet updates itself with the correct 
number at the bottom

int name=Light Wood270/int
int name=Dark Wood68/int
int name=Medium Wood66/int

If this were the same number pre and post filter query I would assume that it 
was simply my data that was bad, however I've pored over this for the better 
part of a day and I'm pretty sure it's the plugin. For reference, this field 
that I'm faceting on is a multiValued field, however I have noticed the exact 
same behavior on non multiValued fields (such as price). 

I can provide any other details you might need



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Trappy behavior with default search field

2014-06-05 Thread Alexandre Rafalovitch

How about a warning section in Admin UI that shows possible issues
with the configuration.

Could start with something absolutely basic like missing default field
(we can detect that from configuration, right?) but I am sure there
are other issues we could think of.

Regards,
   Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency


On Thu, Jun 5, 2014 at 11:48 PM, Erick Erickson erickerick...@gmail.com wrote:
 We've all been bitten by the trappy problem with removing the text field
 from schema.xml and then Solr failing to start b/c various handlers
 specifically call it out.

 Problem is that as people build out (particularly) SolrCloud clusters, this
 innocent-seeming action is getting harder and harder to track down.

 Is it worth a JIRA to address? And any clues how to address it? I started
 thinking about in a _very_ superficial manner and I suspect that this is one
 of those things that _seems_ easy but turns into a sticky wicket.

 If we make sure and return a field that _is_ defined, then the problem
 becomes even harder to detect. I mean you don't even get any warning but
 don't find your docs b/c the default field isn't there and you're searching
 on a different field than you think. At least the current behavior sometimes
 causes Solr to not start at all.

 Im not even sure it's worth doing, we currently both print an error in the
 log and return an error message for a search, but wanted to gather other's
 thoughts.





-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018987#comment-14018987
 ] 

Michael McCandless commented on LUCENE-5738:


Simple (non-Lucene) Java test repros the issue:

{noformat}

import java.io.File;
import java.io.IOException;

import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;

import java.nio.channels.OverlappingFileLockException;
import java.nio.file.StandardOpenOption;

public class TestLock {
public static void obtain() throws Exception {
File path = new File(/tmp/LOCK);
FileChannel channel = FileChannel.open(path.toPath(), 
StandardOpenOption.CREATE, StandardOpenOption.WRITE);
System.out.println(got channel  + channel);
FileLock lock = null;
try {
lock = channel.tryLock();
} catch (IOException | OverlappingFileLockException e) {
// failed to get lock
} finally {
if (lock == null) {
System.out.println(FAILED);
channel.close();
 } else {
System.out.println(SUCCESS);
}
}
}

public static void main(String[] foo) throws Exception {
obtain();
obtain();
Thread.sleep(Integer.MAX_VALUE);
}
}
{noformat}


 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018988#comment-14018988
 ] 

Michael McCandless commented on LUCENE-5738:


[~thetaphi] I think the JVM is still buggy here, and we need that hacky static 
map back maybe?

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018990#comment-14018990
 ] 

Michael McCandless commented on LUCENE-5738:


Also it's really too bad our tests didn't detect this.

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5739) Add zig-zag encoding support to DataInput/DataOutput

Adrien Grand created LUCENE-5739:


 Summary: Add zig-zag encoding support to DataInput/DataOutput
 Key: LUCENE-5739
 URL: https://issues.apache.org/jira/browse/LUCENE-5739
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Fix For: 4.9, 5.0


It would be convenient to have support for zig-zag-encoded integers in 
DataInput/DataOutput. There are not many places that use that feature today but 
I think it's quite common to need to read/write small signed values.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5739) Add zig-zag encoding support to DataInput/DataOutput


 [ 
https://issues.apache.org/jira/browse/LUCENE-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-5739:
-

Attachment: LUCENE-5739.patch

Here is a patch.

 Add zig-zag encoding support to DataInput/DataOutput
 

 Key: LUCENE-5739
 URL: https://issues.apache.org/jira/browse/LUCENE-5739
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5739.patch


 It would be convenient to have support for zig-zag-encoded integers in 
 DataInput/DataOutput. There are not many places that use that feature today 
 but I think it's quite common to need to read/write small signed values.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5739) Add zig-zag encoding support to DataInput/DataOutput

2014-06-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019029#comment-14019029
 ] 

Adrien Grand commented on LUCENE-5739:
--

I know there is a long history of discussions around supporting negative values 
in {{writeVLong}} but I think it would be nice so that the whole long range 
could be readable/writable with zig-zag encoding?

 Add zig-zag encoding support to DataInput/DataOutput
 

 Key: LUCENE-5739
 URL: https://issues.apache.org/jira/browse/LUCENE-5739
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5739.patch


 It would be convenient to have support for zig-zag-encoded integers in 
 DataInput/DataOutput. There are not many places that use that feature today 
 but I think it's quite common to need to read/write small signed values.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Trappy behavior with default search field

2014-06-05 Thread Shawn Heisey

On 6/5/2014 10:48 AM, Erick Erickson wrote:
 We've all been bitten by the trappy problem with removing the text
 field from schema.xml and then Solr failing to start b/c various
 handlers specifically call it out.

 Problem is that as people build out (particularly) SolrCloud clusters,
 this innocent-seeming action is getting harder and harder to track down.

 Is it worth a JIRA to address? And any clues how to address it? I
 started thinking about in a _very_ superficial manner and I suspect
 that this is one of those things that _seems_ easy but turns into a
 sticky wicket.

 If we make sure and return a field that _is_ defined, then the problem
 becomes even harder to detect. I mean you don't even get any warning
 but don't find your docs b/c the default field isn't there and you're
 searching on a different field than you think. At least the current
 behavior sometimes causes Solr to not start at all.

 Im not even sure it's worth doing, we currently both print an error in
 the log and return an error message for a search, but wanted to gather
 other's thoughts.

The minimum action here would be to put a comment on the text field in
the example schema alerting users to the fact that this field is
referenced in the example config as the default field, so removing it
will require config changes.

I've wondered whether we need to change the name of that field so it
actually contains the word default.  A name of default_text or
defaultText would make the problem easier for a user to understand when
they look at their logs after removing the field.

I think the problem that happens when a user types a query like
somefield:abc 123 is much harder.  If it's possible to detect this
specific kind of query (a search against a specific field, followed by a
space and a bare term), we could log a warning.  We'd need to provide a
config option to turn that warning off.  The warning should probably
reference a wiki or cwiki page explaining what the warning means and how
to disable it.  If we did this, the config option to disable the warning
should be in the example config, commented out.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers


[ 
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019045#comment-14019045
 ] 

ASF subversion and git services commented on LUCENE-5703:
-

Commit 1600716 from [~rcmuir] in branch 'dev/trunk'
[ https://svn.apache.org/r1600716 ]

LUCENE-5703: fix safety bug for FC's BINARY too

 Don't allocate/copy bytes all the time in binary DV producers
 -

 Key: LUCENE-5703
 URL: https://issues.apache.org/jira/browse/LUCENE-5703
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, 
 LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch


 Our binary doc values producers keep on creating new {{byte[]}} arrays and 
 copying bytes when a value is requested, which likely doesn't help 
 performance. This has been done because of the way fieldcache consumers used 
 the API, but we should try to fix it in 5.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Trappy behavior with default search field

2014-06-05 Thread david.w.smi...@gmail.com

In my view, solrconfig.xml shouldn’t refer to any field by name out of the
box, except for the /browse handler, and perhaps pre-filling the query form
in the admin GUI.  That’s it.

A couple years ago at about the time I became a committer, I finally did
something about a feature I am very opinionated about something I hate (and
there are few things I hate; ‘qt’ is another) — specifying the default
search field and default operator in the schema. So thankfully it’s
commented out and deprecated now. I ideally would have have gone farther
such that the default solrconfig.xml doesn’t set “df” in /select.  Hoss
added that because, if I recall, some tests broke, amongst other possible
reasons.  In my view, the only reason to keep “df” pre-configured in
/select is for back-wards compatibility expectations with sample queries on
tutorials/websites, but I’d like to see it done for 5x at least.

Furthermore, in my view, “df” (and q.op) should only be “seen” as a
local-param, with the further modification that all top-level parameters
can become virtually local-params to the ‘q’ param.  I should be able to
write a “fq”, “facet.query”, or one of the other myriad of queries using
standard default Lucene syntax without a default field and with the
operator being assumed OR unless I locally change it in local-params.
 Doing otherwise is an ambiguous query if looking at the query by itself.

~ David

On Thu, Jun 5, 2014 at 12:48 PM, Erick Erickson erickerick...@gmail.com
wrote:

 We've all been bitten by the trappy problem with removing the text field
 from schema.xml and then Solr failing to start b/c various handlers
 specifically call it out.

 Problem is that as people build out (particularly) SolrCloud clusters,
 this innocent-seeming action is getting harder and harder to track down.

 Is it worth a JIRA to address? And any clues how to address it? I started
 thinking about in a _very_ superficial manner and I suspect that this is
 one of those things that _seems_ easy but turns into a sticky wicket.

 If we make sure and return a field that _is_ defined, then the problem
 becomes even harder to detect. I mean you don't even get any warning but
 don't find your docs b/c the default field isn't there and you're searching
 on a different field than you think. At least the current behavior
 sometimes causes Solr to not start at all.

 Im not even sure it's worth doing, we currently both print an error in the
 log and return an error message for a search, but wanted to gather other's
 thoughts.

[jira] [Commented] (SOLR-6088) Add query re-ranking with the ReRankingQParserPlugin

2014-06-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019061#comment-14019061
 ] 

ASF subversion and git services commented on SOLR-6088:
---

Commit 1600720 from [~joel.bernstein] in branch 'dev/trunk'
[ https://svn.apache.org/r1600720 ]

SOLR-6088: Add query re-ranking with the ReRankingQParserPlugin

 Add query re-ranking with the ReRankingQParserPlugin
 

 Key: SOLR-6088
 URL: https://issues.apache.org/jira/browse/SOLR-6088
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Joel Bernstein
 Attachments: SOLR-6088.patch, SOLR-6088.patch, SOLR-6088.patch, 
 SOLR-6088.patch, SOLR-6088.patch


 This ticket introduces the ReRankingQParserPlugin which adds query 
 Reranking/Rescoring for Solr. It leverages the new RankQuery framework to 
 plug-in the new Lucene QueryRescorer.
 See ticket LUCENE-5489 for details on the use case.
 Sample syntax:
 {code}
 q={!rerank mainQuery=$qq reRankQuery=$rqq reRankDocs=200}
 {code}
 In the example above the mainQuery is executed and 200 docs are collected and 
 re-ranked based on the results of the reRankQuery. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed

2014-06-05 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer reassigned LUCENE-5738:
---

Assignee: Simon Willnauer

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers

2014-06-05 Thread Robert Muir (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir resolved LUCENE-5703.
-

Resolution: Fixed

Thanks Adrien!

 Don't allocate/copy bytes all the time in binary DV producers
 -

 Key: LUCENE-5703
 URL: https://issues.apache.org/jira/browse/LUCENE-5703
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, 
 LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch


 Our binary doc values producers keep on creating new {{byte[]}} arrays and 
 copying bytes when a value is requested, which likely doesn't help 
 performance. This has been done because of the way fieldcache consumers used 
 the API, but we should try to fix it in 5.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5703) Don't allocate/copy bytes all the time in binary DV producers


[ 
https://issues.apache.org/jira/browse/LUCENE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019137#comment-14019137
 ] 

ASF subversion and git services commented on LUCENE-5703:
-

Commit 1600741 from [~rcmuir] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1600741 ]

LUCENE-5703: BinaryDocValues producers don't allocate or copy bytes on each 
access anymore

 Don't allocate/copy bytes all the time in binary DV producers
 -

 Key: LUCENE-5703
 URL: https://issues.apache.org/jira/browse/LUCENE-5703
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
 Fix For: 4.9, 5.0

 Attachments: LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, 
 LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch, LUCENE-5703.patch


 Our binary doc values producers keep on creating new {{byte[]}} arrays and 
 copying bytes when a value is requested, which likely doesn't help 
 performance. This has been done because of the way fieldcache consumers used 
 the API, but we should try to fix it in 5.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5721) Monotonic packed could maybe be faster

2014-06-05 Thread ASF subversion and git services (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019178#comment-14019178
]

ASF subversion and git services commented on LUCENE-5721:
-

Commit 1600747 from [~jpountz] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1600747 ]

LUCENE-5721: Monotonic compression doesn't use zig-zag encoding anymore.

Monotonic packed could maybe be faster
--

Key: LUCENE-5721
URL: https://issues.apache.org/jira/browse/LUCENE-5721
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
Assignee: Adrien Grand
Attachments: LUCENE-5703.patch

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5721) Monotonic packed could maybe be faster

2014-06-05 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-5721.
--

Resolution: Fixed

 Monotonic packed could maybe be faster
 --

 Key: LUCENE-5721
 URL: https://issues.apache.org/jira/browse/LUCENE-5721
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Robert Muir
Assignee: Adrien Grand
 Attachments: LUCENE-5703.patch


 This compression is used in lucene for monotonically increasing offsets, e.g. 
 stored fields index, dv BINARY/SORTED_SET offsets, OrdinalMap (used for 
 merging and faceting dv) and so on.
 Today this stores a +/- deviation from an expected line of y=mx + b, where b 
 is the minValue for the block and m is the average delta from the previous 
 value. Because it can be negative, we have to do some additional work to 
 zigzag-decode.
 Can we just instead waste a bit for every value explicitly (lower the 
 minValue by the min delta) so that deltas are always positive and we can have 
 a simpler decode? Maybe If we do this, the new guy should assert that values 
 are actually monotic at write-time. The current one supports mostly monotic 
 but do we really need that flexibility anywhere? If so it could always be 
 kept...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-6139) Update specific field on SOLR index in java codeing

2014-06-05 Thread Erick Erickson (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson closed SOLR-6139.


Resolution: Duplicate

see: https://issues.apache.org/jira/browse/LUCENE-4258

This is still unresolved.

In future, please raise issues on the dev and/or user's list before making a 
JIRA to see what prior art has been done.

 Update specific field on SOLR index in java codeing 
 

 Key: SOLR-6139
 URL: https://issues.apache.org/jira/browse/SOLR-6139
 Project: Solr
  Issue Type: Improvement
Reporter: muruganv
   Original Estimate: 24h
  Remaining Estimate: 24h

 Update specific field on SOLR index in java codeing.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6144) DIH Cache backed with MapDB

2014-06-05 Thread James Dyer (JIRA)

James Dyer created SOLR-6144:


 Summary: DIH Cache backed with MapDB
 Key: SOLR-6144
 URL: https://issues.apache.org/jira/browse/SOLR-6144
 Project: Solr
  Issue Type: New Feature
  Components: contrib - DataImportHandler
Reporter: James Dyer
Priority: Minor


This is a DIH Cache implementation using Map DB version 1.0.2.  Map DB is a 
pure Java key-value store that supports both file-based caches and memory-based 
caches (both on- and off-heap).  See http://www.mapdb.org/.  MapDB is ASL2.0 
licensed, so unlike the BDB-JE cache impl (SOLR-2613), this one could 
potentially be committed.

This does not perform nearly as well as the BDB-JE cache, but I imagine it is 
fast enough for a lot of uses.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6144) DIH Cache backed with MapDB

2014-06-05 Thread James Dyer (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Dyer updated SOLR-6144:
-

Attachment: SOLR-6144.patch

The attached patch requires SOLR-2943 (patch, uncommitted) also.

 DIH Cache backed with MapDB
 ---

 Key: SOLR-6144
 URL: https://issues.apache.org/jira/browse/SOLR-6144
 Project: Solr
  Issue Type: New Feature
  Components: contrib - DataImportHandler
Reporter: James Dyer
Priority: Minor
 Attachments: SOLR-6144.patch


 This is a DIH Cache implementation using Map DB version 1.0.2.  Map DB is a 
 pure Java key-value store that supports both file-based caches and 
 memory-based caches (both on- and off-heap).  See http://www.mapdb.org/.  
 MapDB is ASL2.0 licensed, so unlike the BDB-JE cache impl (SOLR-2613), this 
 one could potentially be committed.
 This does not perform nearly as well as the BDB-JE cache, but I imagine it is 
 fast enough for a lot of uses.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5740) Add stripContentOfTags option to HTMLStripCharFilter

2014-06-05 Thread David Smiley (JIRA)

David Smiley created LUCENE-5740:


 Summary: Add stripContentOfTags option to HTMLStripCharFilter
 Key: LUCENE-5740
 URL: https://issues.apache.org/jira/browse/LUCENE-5740
 Project: Lucene - Core
  Issue Type: New Feature
  Components: modules/analysis
Reporter: David Smiley


HTMLStripCharFilter should have an option to strip out the sub-content of 
certain elements. It already does this for SCRIPT  STYLE but it should be 
configurable to add more.  I don't want certain elements to have their contents 
to be searchable.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6088) Add query re-ranking with the ReRankingQParserPlugin


[ 
https://issues.apache.org/jira/browse/SOLR-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019268#comment-14019268
 ] 

ASF subversion and git services commented on SOLR-6088:
---

Commit 1600760 from [~joel.bernstein] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1600760 ]

SOLR-6088: Add query re-ranking with the ReRankingQParserPlugin

 Add query re-ranking with the ReRankingQParserPlugin
 

 Key: SOLR-6088
 URL: https://issues.apache.org/jira/browse/SOLR-6088
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Joel Bernstein
 Attachments: SOLR-6088.patch, SOLR-6088.patch, SOLR-6088.patch, 
 SOLR-6088.patch, SOLR-6088.patch


 This ticket introduces the ReRankingQParserPlugin which adds query 
 Reranking/Rescoring for Solr. It leverages the new RankQuery framework to 
 plug-in the new Lucene QueryRescorer.
 See ticket LUCENE-5489 for details on the use case.
 Sample syntax:
 {code}
 q={!rerank mainQuery=$qq reRankQuery=$rqq reRankDocs=200}
 {code}
 In the example above the mainQuery is executed and 200 docs are collected and 
 re-ranked based on the results of the reRankQuery. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5734) HTMLStripCharFilter end offset should be left of closing tags

2014-06-05 Thread David Smiley (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019271#comment-14019271
 ] 

David Smiley commented on LUCENE-5734:
--

Simply re-stating my expectation of how it should work from IRC:
My expectation is that the startOffset and endOffset of a token should be 
directly adjacent to the token text in the original text.  This is the case for 
startOffset, but with HTMLStripCharFilter (and maybe others?) endOffset isn't; 
it follows elided (stripped) characters.

I don't think there's a rule it should be any which way, but I claim it should 
work this way -- FWIW I think it's intuitive.  Never mind what I said about it 
being sensitive to an inner opening tag; I can live without that complication.




 HTMLStripCharFilter end offset should be left of closing tags
 -

 Key: LUCENE-5734
 URL: https://issues.apache.org/jira/browse/LUCENE-5734
 Project: Lucene - Core
  Issue Type: Bug
  Components: modules/analysis
Reporter: David Smiley
Priority: Minor

 Consider this simple input:
 {noformat}
 emhello/em
 {noformat}
 to be analyzed by HTMLStripCharFilter and WhitespaceTokenizer.
 You get back one token for hello.  Good.  The start offset of this token is 
 at the position of 'h' -- good.  But the end offset is surprisingly plus one 
 to the adjacent /em.  I argue that it should be plus one to the last 
 character of the token (following 'o').
 FYI it behaves as I expect if after hello is an XML entity such as in this 
 example: {noformat}hellonbsp;{noformat} The end offset immediately follows 
 the 'o'.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6088) Add query re-ranking with the ReRankingQParserPlugin

2014-06-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019277#comment-14019277
 ] 

ASF subversion and git services commented on SOLR-6088:
---

Commit 1600765 from [~joel.bernstein] in branch 'dev/trunk'
[ https://svn.apache.org/r1600765 ]

SOLR-6088: Add query re-ranking with the ReRankingQParserPlugin

 Add query re-ranking with the ReRankingQParserPlugin
 

 Key: SOLR-6088
 URL: https://issues.apache.org/jira/browse/SOLR-6088
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Joel Bernstein
 Attachments: SOLR-6088.patch, SOLR-6088.patch, SOLR-6088.patch, 
 SOLR-6088.patch, SOLR-6088.patch


 This ticket introduces the ReRankingQParserPlugin which adds query 
 Reranking/Rescoring for Solr. It leverages the new RankQuery framework to 
 plug-in the new Lucene QueryRescorer.
 See ticket LUCENE-5489 for details on the use case.
 Sample syntax:
 {code}
 q={!rerank mainQuery=$qq reRankQuery=$rqq reRankDocs=200}
 {code}
 In the example above the mainQuery is executed and 200 docs are collected and 
 re-ranked based on the results of the reRankQuery. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6088) Add query re-ranking with the ReRankingQParserPlugin

2014-06-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019281#comment-14019281
 ] 

ASF subversion and git services commented on SOLR-6088:
---

Commit 1600767 from [~joel.bernstein] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1600767 ]

SOLR-6088: Add query re-ranking with the ReRankingQParserPlugin

 Add query re-ranking with the ReRankingQParserPlugin
 

 Key: SOLR-6088
 URL: https://issues.apache.org/jira/browse/SOLR-6088
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Joel Bernstein
 Attachments: SOLR-6088.patch, SOLR-6088.patch, SOLR-6088.patch, 
 SOLR-6088.patch, SOLR-6088.patch


 This ticket introduces the ReRankingQParserPlugin which adds query 
 Reranking/Rescoring for Solr. It leverages the new RankQuery framework to 
 plug-in the new Lucene QueryRescorer.
 See ticket LUCENE-5489 for details on the use case.
 Sample syntax:
 {code}
 q={!rerank mainQuery=$qq reRankQuery=$rqq reRankDocs=200}
 {code}
 In the example above the mainQuery is executed and 200 docs are collected and 
 re-ranked based on the results of the reRankQuery. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6088) Add query re-ranking with the ReRankingQParserPlugin

2014-06-05 Thread Joel Bernstein (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-6088:
-

Description: 
This ticket introduces the ReRankingQParserPlugin which adds query 
Reranking/Rescoring for Solr. It leverages the new RankQuery framework to 
plug-in the new Lucene QueryRescorer.

See ticket LUCENE-5489 for details on the use case.

Sample syntax:
{code}
q=*:*rq={!rerank reRankQuery=$rqq reRankDocs=200 reRankWeight=3}
{code}
In the example above the mainQuery is executed and 200 docs are collected and 
re-ranked based on the results of the reRankQuery. 





  was:
This ticket introduces the ReRankingQParserPlugin which adds query 
Reranking/Rescoring for Solr. It leverages the new RankQuery framework to 
plug-in the new Lucene QueryRescorer.

See ticket LUCENE-5489 for details on the use case.

Sample syntax:
{code}
q={!rerank mainQuery=$qq reRankQuery=$rqq reRankDocs=200}
{code}
In the example above the mainQuery is executed and 200 docs are collected and 
re-ranked based on the results of the reRankQuery. 






 Add query re-ranking with the ReRankingQParserPlugin
 

 Key: SOLR-6088
 URL: https://issues.apache.org/jira/browse/SOLR-6088
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Joel Bernstein
 Attachments: SOLR-6088.patch, SOLR-6088.patch, SOLR-6088.patch, 
 SOLR-6088.patch, SOLR-6088.patch


 This ticket introduces the ReRankingQParserPlugin which adds query 
 Reranking/Rescoring for Solr. It leverages the new RankQuery framework to 
 plug-in the new Lucene QueryRescorer.
 See ticket LUCENE-5489 for details on the use case.
 Sample syntax:
 {code}
 q=*:*rq={!rerank reRankQuery=$rqq reRankDocs=200 reRankWeight=3}
 {code}
 In the example above the mainQuery is executed and 200 docs are collected and 
 re-ranked based on the results of the reRankQuery. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1622 - Failure!

2014-06-05 Thread Dawid Weiss

Background thread initiating backup hung on opening the connection to
the server (there's a buffered input stream that attempts to fill()
its buffer). The master test thread never waits for those threads to
finish.

This test is broken in more ways than one...

Dawid



On Thu, Jun 5, 2014 at 3:36 PM, Policeman Jenkins Server
jenk...@thetaphi.de wrote:
 Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1622/
 Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

 2 tests failed.
 FAILED:  
 junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

 Error Message:
 1 thread leaked from SUITE scope at 
 org.apache.solr.handler.TestReplicationHandlerBackup: 1) Thread[id=1179, 
 name=Thread-381, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] 
 at java.net.SocketInputStream.socketRead0(Native Method) at 
 java.net.SocketInputStream.read(SocketInputStream.java:150) at 
 java.net.SocketInputStream.read(SocketInputStream.java:121) at 
 java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at 
 java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at 
 java.io.BufferedInputStream.read(BufferedInputStream.java:345) at 
 sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:701) at 
 sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) at 
 sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1534)
  at 
 sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
  at java.net.URL.openStream(URL.java:1038) at 
 org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:328)

 Stack Trace:
 com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from 
 SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:
1) Thread[id=1179, name=Thread-381, state=RUNNABLE, 
 group=TGRP-TestReplicationHandlerBackup]
 at java.net.SocketInputStream.socketRead0(Native Method)
 at java.net.SocketInputStream.read(SocketInputStream.java:150)
 at java.net.SocketInputStream.read(SocketInputStream.java:121)
 at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
 at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
 at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
 at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:701)
 at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
 at 
 sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1534)
 at 
 sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
 at java.net.URL.openStream(URL.java:1038)
 at 
 org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:328)
 at __randomizedtesting.SeedInfo.seed([69441328FAB793D3]:0)


 FAILED:  
 junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

 Error Message:
 There are still zombie threads that couldn't be terminated:1) 
 Thread[id=1179, name=Thread-381, state=RUNNABLE, 
 group=TGRP-TestReplicationHandlerBackup] at 
 java.net.SocketInputStream.socketRead0(Native Method) at 
 java.net.SocketInputStream.read(SocketInputStream.java:150) at 
 java.net.SocketInputStream.read(SocketInputStream.java:121) at 
 java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at 
 java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at 
 java.io.BufferedInputStream.read(BufferedInputStream.java:345) at 
 sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:701) at 
 sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) at 
 sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1534)
  at 
 sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
  at java.net.URL.openStream(URL.java:1038) at 
 org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:328)

 Stack Trace:
 com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
 threads that couldn't be terminated:
1) Thread[id=1179, name=Thread-381, state=RUNNABLE, 
 group=TGRP-TestReplicationHandlerBackup]
 at java.net.SocketInputStream.socketRead0(Native Method)
 at java.net.SocketInputStream.read(SocketInputStream.java:150)
 at java.net.SocketInputStream.read(SocketInputStream.java:121)
 at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
 at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
 at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
 at

[jira] [Updated] (SOLR-6088) Add query re-ranking with the ReRankingQParserPlugin

2014-06-05 Thread Joel Bernstein (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-6088:
-

Fix Version/s: 4.9

 Add query re-ranking with the ReRankingQParserPlugin
 

 Key: SOLR-6088
 URL: https://issues.apache.org/jira/browse/SOLR-6088
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Joel Bernstein
 Fix For: 4.9

 Attachments: SOLR-6088.patch, SOLR-6088.patch, SOLR-6088.patch, 
 SOLR-6088.patch, SOLR-6088.patch


 This ticket introduces the ReRankingQParserPlugin which adds query 
 Reranking/Rescoring for Solr. It leverages the new RankQuery framework to 
 plug-in the new Lucene QueryRescorer.
 See ticket LUCENE-5489 for details on the use case.
 Sample syntax:
 {code}
 q=*:*rq={!rerank reRankQuery=$rqq reRankDocs=200 reRankWeight=3}
 {code}
 In the example above the mainQuery is executed and 200 docs are collected and 
 re-ranked based on the results of the reRankQuery. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed

2014-06-05 Thread Uwe Schindler (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019319#comment-14019319
 ] 

Uwe Schindler commented on LUCENE-5738:
---

bq. Uwe Schindler I think the JVM is still buggy here, and we need that hacky 
static map back maybe?

The static map was there to not obtain the lock 2 times from the same JVM. It 
could help to redo this, but I am not yet sure, what the real bug is. Does this 
also release the lock of another process? If yes, its a problem of the O/S.

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5734) HTMLStripCharFilter end offset should be left of closing tags


[ 
https://issues.apache.org/jira/browse/LUCENE-5734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019325#comment-14019325
 ] 

Steve Rowe commented on LUCENE-5734:


Paraphrasing my answer to David from IRC: adjacency doesn't fully describe 
the effect you're looking for, since text is adjacent both before and after 
both opening and closing tags.

Semantics aside, I agree that moving offsets prior to closing tags would align 
better with intuitive expectations, and would very likely reduce the number of 
fixups highlighters would have to make to balance tags for any given snippet 
within marked up text.

My only remaining concern is whether changing the behavior will negatively 
affect existing users.  Maybe we could make the behavior configurable?  If 
that's done, there remains the question of whether to leave the default 
behavior as it is now, or make the default be the new behavior.

 HTMLStripCharFilter end offset should be left of closing tags
 -

 Key: LUCENE-5734
 URL: https://issues.apache.org/jira/browse/LUCENE-5734
 Project: Lucene - Core
  Issue Type: Bug
  Components: modules/analysis
Reporter: David Smiley
Priority: Minor

 Consider this simple input:
 {noformat}
 emhello/em
 {noformat}
 to be analyzed by HTMLStripCharFilter and WhitespaceTokenizer.
 You get back one token for hello.  Good.  The start offset of this token is 
 at the position of 'h' -- good.  But the end offset is surprisingly plus one 
 to the adjacent /em.  I argue that it should be plus one to the last 
 character of the token (following 'o').
 FYI it behaves as I expect if after hello is an XML entity such as in this 
 example: {noformat}hellonbsp;{noformat} The end offset immediately follows 
 the 'o'.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5740) Add stripContentOfTags option to HTMLStripCharFilter


[ 
https://issues.apache.org/jira/browse/LUCENE-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019335#comment-14019335
 ] 

Steve Rowe commented on LUCENE-5740:


+1 - it would likely be very simple to add this functionality.


 Add stripContentOfTags option to HTMLStripCharFilter
 

 Key: LUCENE-5740
 URL: https://issues.apache.org/jira/browse/LUCENE-5740
 Project: Lucene - Core
  Issue Type: New Feature
  Components: modules/analysis
Reporter: David Smiley

 HTMLStripCharFilter should have an option to strip out the sub-content of 
 certain elements. It already does this for SCRIPT  STYLE but it should be 
 configurable to add more.  I don't want certain elements to have their 
 contents to be searchable.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5697) Preview issue

[
https://issues.apache.org/jira/browse/LUCENE-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019350#comment-14019350
]

Hoss Man commented on LUCENE-5697:
--

1) Lucene 3.5 is pretty old.

2) At first glance, it sounds like the problems you are describing could simply
be due to a disconnect between how your searches are executed vs how you are
using the highlighter code.

w/o specific example code or a reproducible test case, there's really no way to
tell if what you are describing is a genuine bug or a missunderstanding of the
API.

3) there multiple highlighters available, and a *lot* of different ways to
configure them, so even if there is a bug, w/o more specifics there really
isn't enough info here to try and diagnose _where_ the bug is, let alone _what_
the bug is.

---

can you please provide some code (ideally a stand alone JUnit test using the
lucene test-framework) demonstrating the problem?

Preview issue
-

Key: LUCENE-5697
URL: https://issues.apache.org/jira/browse/LUCENE-5697
Project: Lucene - Core
Issue Type: Bug
Components: modules/highlighter
Environment: DocFetcher 1.1.11 on Win 7(64) pro
Reporter: Martin Schoenmakers

In DocFetcher, which uses Lucene v3.5.0, we stumbled on a bug. The lead of
DocFetcher has investigated and found the problem seems to be in Lucene. I do
not know if this bug has been fixed in a later Lucene version.
Issue:
We use proximity search: search on multiple words in a directory with about
300 PDF files.
E.g. search for wordA wordB wordC~50, i.e. three words within 50 words
distance of each other. The resulting documents are correct. But the
highligted text in the document is often missing.
If the words are in the SAME order as in the search AND on the SAME page,
then the higlight works correct. But if the order of the words is different
from the search (like wordA wordC wordB OR the words are not on the same
page, then that text is not highlighted.
As we use the proximity search on multiple words often, it severely degrades
the usability.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5734) HTMLStripCharFilter end offset should be left of closing tags


[ 
https://issues.apache.org/jira/browse/LUCENE-5734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019355#comment-14019355
 ] 

Steve Rowe commented on LUCENE-5734:


bq. FYI it behaves as I expect if after hello is an XML entity such as in this 
example: {{helloamp;nbsp;}}

I should point out that HTMLStripCharFilter only accepts the named character 
entities defined in [the HTML 4.0 
spec|http://www.w3.org/TR/REC-html40/sgml/entities.html], which happens to 
include the predefined XML entities ({{amp;gt;}}, {{amp;lt;}}, 
{{amp;quot;}}, {{amp;apos;}}, and {{amp;amp;}}).  {{amp;nbsp;}} is 
specifically *not* an XML entity, at least not as understood by 
HTMLStripCharFilter.  I mention this only to point out that HTMLStripCharFilter 
doesn't parse XML for entity declarations, and will not honor them if they 
appear.

 HTMLStripCharFilter end offset should be left of closing tags
 -

 Key: LUCENE-5734
 URL: https://issues.apache.org/jira/browse/LUCENE-5734
 Project: Lucene - Core
  Issue Type: Bug
  Components: modules/analysis
Reporter: David Smiley
Priority: Minor

 Consider this simple input:
 {noformat}
 emhello/em
 {noformat}
 to be analyzed by HTMLStripCharFilter and WhitespaceTokenizer.
 You get back one token for hello.  Good.  The start offset of this token is 
 at the position of 'h' -- good.  But the end offset is surprisingly plus one 
 to the adjacent /em.  I argue that it should be plus one to the last 
 character of the token (following 'o').
 FYI it behaves as I expect if after hello is an XML entity such as in this 
 example: {noformat}hellonbsp;{noformat} The end offset immediately follows 
 the 'o'.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6113) Edismax doesn't parse well the query uf (User Fields)


[ 
https://issues.apache.org/jira/browse/SOLR-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019356#comment-14019356
 ] 

Hoss Man commented on SOLR-6113:


bq. I like the idea of removing just the field name, because I agree it gives a 
better result when I look at it from the permission use case scenario

I think you are overlooking the designed usecase: colons appearing in natural 
text should be treated as regular text *unless* they are explicitly noted as 
special via the uf option.

As yonik pointed out in some recent examples on the mailing list, if a user 
searches for {{Mission: Impossible}} we should not ignore the {{Mission}} 
portion of the query string if there is no Mission field name and no 
explicitly configured Mission field alias.

the default assumption is that the query string is raw query words -- only if 
the uf options tell us to treat some text as special (either a field name 
or a field alias) should it be treated as special syntax to mean search in 
this field or serach against this field alias


 Edismax doesn't parse well the query uf (User Fields)
 -

 Key: SOLR-6113
 URL: https://issues.apache.org/jira/browse/SOLR-6113
 Project: Solr
  Issue Type: Improvement
  Components: query parsers
Reporter: Liram Vardi
Priority: Minor
  Labels: edismax

 It seems that Edismax User Fields feature does not behave as expected.
 For instance, assuming the following query:
 _q= id:b* user:Anna CollinsdefType=edismaxuf=* -userrows=0_
 The parsed query (taken from query debug info) is:
 _+((id:b* (text:user) (text:anna collins))~1)_
 I expect that because user was filtered out in uf (User fields), the 
 parsed query should not contain the user search part.
 In another words, the parsed query should look simply like this:  _+id:b*_
 This issue is affected by a the patch on issue SOLR-2649: When changing the 
 default OP of Edismax to AND, the query results change.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019395#comment-14019395
 ] 

Michael McCandless commented on LUCENE-5738:


Somehow, the 2nd channel.close releases the first lock obtained in the same 
process.

I.e., if you start that test program you'll see it print SUCCESS then FAILED, 
and while it's still running go run it again in another window, and that 2nd 
process also prints SUCCESS then FAILED.

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6137) Managed Schema / Schemaless and SolrCloud concurrency issues

[
https://issues.apache.org/jira/browse/SOLR-6137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019403#comment-14019403
]

Steve Rowe commented on SOLR-6137:
--

bq. making multiple schema API calls concurrently can block; that is, one may
get through and the other may infinite loop.

[~alexey] mentioned that problem to me a while back. IIRC this is a bug in the
optimistic concurrency implementation. I'll dig up his info and make an issue.

bq. Schema API changes return success before all cores are updated; subsequent
calls attempting to use new schema may fail

One way to fix this may be to return the schema ZK node version from Schema API
modification methods, and add a request param requiring a minimum schema ZK
node version prior to binding a schema snapshot to the request.

bq. no blocking mode should be added to schemaless... that's what the
optimistic concurrency is for.

Right, thanks Yonik, I'd forgotten optimistic concurrency's part in the
schemaless design.

Managed Schema / Schemaless and SolrCloud concurrency issues

Key: SOLR-6137
URL: https://issues.apache.org/jira/browse/SOLR-6137
Project: Solr
Issue Type: Bug
Components: Schema and Analysis, SolrCloud
Reporter: Gregory Chanan

This is a follow up to a message on the mailing list, linked here:
http://mail-archives.apache.org/mod_mbox/lucene-dev/201406.mbox/%3CCAKfebOOcMeVEb010SsdcH8nta%3DyonMK5R7dSFOsbJ_tnre0O7w%40mail.gmail.com%3E
The Managed Schema integration with SolrCloud seems pretty limited.
The issue I'm running into is variants of the issue that schema changes are
not pushed to all shards/replicas synchronously. So, for example, I can make
the following two requests:
1) add a field to the collection on server1 using the Schema API
2) add a document with the new field, the document is routed to a core on
server2
Then, there appears to be a race between when the document is processed by
the core on server2 and when the core on server2, via the
ZkIndexSchemaReader, gets the new schema. If the document is processed
first, I get a 400 error because the field doesn't exist. This is easily
reproducible by adding a sleep to the ZkIndexSchemaReader's processing.
I hit a similar issue with Schemaless: the distributed request handler sends
out the document updates, but there is no guarantee that the other
shards/replicas see the schema changes made by the update.chain.
Another issue I noticed today: making multiple schema API calls concurrently
can block; that is, one may get through and the other may infinite loop.
So, for reference, the issues include:
1) Schema API changes return success before all cores are updated; subsequent
calls attempting to use new schema may fail
2) Schemaless changes may fail on replicas/other shards for the same reason
3) Concurrent Schema API changes may block
From Steve Rowe on the mailing list:
{quote}
For Schema API users, delaying a couple of seconds after adding fields before
using them should workaround this problem. While not ideal, I think schema
field additions are rare enough in the Solr collection lifecycle that this is
not a huge problem.
For schemaless users, the picture is worse, as you noted. Immediate
distribution of documents triggering schema field addition could easily prove
problematic. Maybe we need a schema update blocking mode, where after the ZK
schema node watch is triggered, all new request processing is halted until
the schema is finished downloading/parsing/swapping out? (Such a mode should
help Schema API users too.)
{quote}

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-6132) UpdateRequest contains only XML ContentStream and not JavaBin


 [ 
https://issues.apache.org/jira/browse/SOLR-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-6132.


Resolution: Not a Problem

{{wt=javabin}} is a request parameter that instructs the server that you want 
the Writer Type used for generating the _response_ to be in javabin.

It has nothing to do with what format your SolrJ code uses when sending the 
_request_.  Yo uare free to send your update data in one format, and request 
that solr send you the response in another.

As a concrete example, imagine if you were using {{ContentStreamUpdateRequest}} 
to stream a bunch of raw binary files to the extracting request handler -- just 
because you set the {{qt}} param to control whether  the result status info is 
returned in javabin, or xml, or json doesn't mean that SolrJ can/should muck 
with the data being sent to solr.



 UpdateRequest contains only XML ContentStream and not JavaBin 
 --

 Key: SOLR-6132
 URL: https://issues.apache.org/jira/browse/SOLR-6132
 Project: Solr
  Issue Type: Improvement
  Components: clients - java, update
Reporter: Liram Vardi
  Labels: UpdateProcessor, Updating, javabincodec, solrj

 When creating a UpdateRequest using the following code, I noted that even 
 though the request params include wt=javabin, the final request is being 
 translated to XML.
 I guess that this is because that the collection of ContentStreams that is 
 returned by UpdateRequest.getContentStreams() method contains only XML 
 ContentStream. 
  
 Should not that UpdateRequest contain JavaBin ContentStream by default or 
 when it gets some parameter (such wt=javabin)?  
 The code:
  UpdateRequest updateRequest = new UpdateRequest();
  updateRequest.add(solrDocument);
  updateRequest.setCommitWithin(-1);
  SolrRequestParsers _parser = new SolrRequestParsers(null);
  SolrQueryRequest req;
  try {
req = _parser.buildRequestFrom(targetCore, params,   
 {color:red}updateRequest.getContentStreams(){color});
  } catch (Exception e) {
   throw new SolrServerException(e);
  }



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-trunk-Linux-java7-64-analyzers - Build # 7199 - Failure!

2014-06-05 Thread builder

Build: builds.flonkings.com/job/Lucene-trunk-Linux-java7-64-analyzers/7199/

No tests ran.

Build Log:
[...truncated 14 lines...]
ERROR: Failed to update http://svn.apache.org/repos/asf/lucene/dev/trunk
org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS 
/repos/asf/lucene/dev/trunk failed
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388)
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373)
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getRepositoryUUID(DAVRepository.java:148)
at 
org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:339)
at 
org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:328)
at 
org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.update(SVNUpdateClient16.java:482)
at 
org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:364)
at 
org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doUpdate(SVNUpdateClient16.java:274)
at 
org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:27)
at 
org.tmatesoft.svn.core.internal.wc2.old.SvnOldUpdate.run(SvnOldUpdate.java:11)
at 
org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
at 
org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at 
org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at 
org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at 
org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at 
hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:157)
at 
hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:910)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:891)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:874)
at hudson.FilePath.act(FilePath.java:914)
at hudson.FilePath.act(FilePath.java:887)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:850)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:788)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1320)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:609)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:518)
at hudson.model.Run.execute(Run.java:1689)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:231)
Caused by: svn: E175002: OPTIONS /repos/asf/lucene/dev/trunk failed
at 
org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
at 
org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
at 
org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
... 38 more
Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS request 
failed on '/repos/asf/lucene/dev/trunk'
svn: E175002: timed out waiting for server
at 
org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
at 
org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:777)
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:382)
... 37 more
Caused by: svn: E175002: OPTIONS request failed on '/repos/asf/lucene/dev/trunk'
at 
org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:775)
... 38 more
Caused by: svn: E175002: timed out waiting for server
at

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed

2014-06-05 Thread Richard Boulton (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019411#comment-14019411
 ] 

Richard Boulton commented on LUCENE-5738:
-

I believe that Java's NativeFSLock uses fcntl on Linux.  Unfortunately, the 
semantics of fcntl are consistent with the behaviour described above.

Specifically:

 - a process has first to open a file descriptor to obtain a lock with fcntl
 - when that file descriptor is closed, the lock is released
 - however, the lock is also released if any file descriptor on the same 
underlying file is closed by the same process as is holding the lock.  (I'm not 
certain of the behaviour when file handles are passed to other processes, 
either explicitly or by forking.)
 - the lock is not released, however, if a different process opens and then 
closes a file descriptor on the file.

So, there are two approaches I know of to get the semantics you probably want 
(ie, that the only way the lock is closed is if the process holding it exits, 
or the lock is explicitly closed).

 1: make sure files which are locked aren't opened multiple times while locked. 
 This probably needs some process-wide state to track which files have locks 
held on them.  This is, of course, awkward for
a library, since you don't have control over code in the same process which 
isn't part of the library.
 2: fork a subprocess to hold the lock open.  This is expensive, but is the 
approach we took with Xapian.  I'm not sure it would be workable if you lock 
things at all frequently, though.

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed

2014-06-05 Thread Richard Boulton (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019417#comment-14019417
 ] 

Richard Boulton commented on LUCENE-5738:
-

In case it wasn't clear in my previous comment, the problem is that when you 
try to obtain a lock on a file already locked by the same process, you open and 
then close a second file descriptor on the file, and when that file descriptor 
is closed, the lock is removed by the OS.

Note this from the FileLock documentation 
(http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/nio/channels/FileLock.java#28
 )

On some systems, closing a channel releases all locks held by the Java virtual 
machine on the underlying file regardless of whether the locks were acquired 
via that channel or via another channel open on the same file. It is strongly 
recommended that, within a program, a unique channel be used to acquire all 
locks on any given file.

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5738) NativeLock is release if Lock is closed after obtain failed


[ 
https://issues.apache.org/jira/browse/LUCENE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019421#comment-14019421
 ] 

Hoss Man commented on LUCENE-5738:
--

bq. Somehow, the 2nd channel.close releases the first lock obtained in the same 
process.

Isn't this a valid aspect of the API as documented?

http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html

{quote}
On some systems, closing a channel releases all locks held by the Java virtual 
machine on the underlying file regardless of whether the locks were acquired 
via that channel or via another channel open on the same file. It is strongly 
recommended that, within a program, a unique channel be used to acquire all 
locks on any given file. 
{quote}

 NativeLock is release if Lock is closed after obtain failed
 ---

 Key: LUCENE-5738
 URL: https://issues.apache.org/jira/browse/LUCENE-5738
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 4.9, 5.0


 if you obtain the NativeFSLock and try to obtain it again in the same JVM and 
 close if if it fails another process will be able to obtain it. This is 
 pretty trappy though. If you execute the main class twice the problem becomes 
 pretty obvious.
 {noformat}
 import org.apache.lucene.store.Lock;
 import org.apache.lucene.store.NativeFSLockFactory;
 import java.io.File;
 import java.io.IOException;
 public class TestLock {
  public static void main(String[] foo) throws IOException, 
 InterruptedException {
 NativeFSLockFactory lockFactory = new NativeFSLockFactory(new 
 File(/tmp));
 Lock lock = lockFactory.makeLock(LOCK);
 if (lock.obtain()) {
 System.out.println(OBTAINED);
 } else {
 lock.close();
 System.out.println(FAILED);
 }
 // try it again and close it if it fails
 lock = lockFactory.makeLock(LOCK); //  this is a new lock
 if (lock.obtain()) {
 System.out.println(OBTAINED AGAIN);
 } else {
 lock.close(); //  this releases the lock we obtained
 System.out.println(FAILED on Second);
 }
 Thread.sleep(Integer.MAX_VALUE);
 }
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6146) Leak in CloudSolrServer causing Too many open files

2014-06-05 Thread Jessica Cheng (JIRA)

Jessica Cheng created SOLR-6146:
---

 Summary: Leak in CloudSolrServer causing Too many open files
 Key: SOLR-6146
 URL: https://issues.apache.org/jira/browse/SOLR-6146
 Project: Solr
  Issue Type: Bug
  Components: clients - java, SolrCloud
Affects Versions: 4.7
Reporter: Jessica Cheng


Due to a misconfiguration in one of our QA clusters, we uncovered a leak in 
CloudSolrServer. If this line throws:

https://github.com/apache/lucene-solr/blob/branch_4x/solr/solrj/src/java/org/apache/solr/client/solrj/impl/CloudSolrServer.java#L242

then the instantiated ZkStateReader is leaked.

Here's the stacktrace of the Exception (we're using a custom build so the line 
numbers won't quite match up, but it gives the idea):
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:304)
 at 
org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:568)
 at 
org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:557)
 at 
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
 at 
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:33)
 at 
com.apple.cie.search.client.crossdc.MirroredSolrRequestHandler.handleItem(MirroredSolrRequestHandler.java:100)
 at 
com.apple.cie.search.client.crossdc.MirroredSolrRequestHandler.handleItem(MirroredSolrRequestHandler.java:33)
 at 
com.apple.coda.queueing.CodaQueueConsumer$StreamProcessor.run(CodaQueueConsumer.java:147)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
at java.lang.Thread.run(Thread.java:722) Caused by: 
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode 
for /live_nodes at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468) at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256) at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253) at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
 at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253) at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:305)
 at 
org.apache.solr.client.solrj.impl.CloudSolrServer.createZkStateReader(CloudSolrServer.java:935)
 at 
org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:298)
 ... 10 more




--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6145) Concurrent Schema API field additions can result in endless loop

Steve Rowe created SOLR-6145:


 Summary: Concurrent Schema API field additions can result in 
endless loop
 Key: SOLR-6145
 URL: https://issues.apache.org/jira/browse/SOLR-6145
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis
Reporter: Steve Rowe
Assignee: Steve Rowe
Priority: Critical


The optimistic concurrency loop in {{ManagedIndexSchema.addFields()}} is the 
likely culprit.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6137) Managed Schema / Schemaless and SolrCloud concurrency issues

[
https://issues.apache.org/jira/browse/SOLR-6137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019432#comment-14019432
]

Steve Rowe commented on SOLR-6137:
--

{quote}
bq. making multiple schema API calls concurrently can block; that is, one may
get through and the other may infinite loop.

[~alexey] mentioned that problem to me a while back. IIRC this is a bug in the
optimistic concurrency implementation. I'll dig up his info and make an issue.
{quote}

SOLR-6145

Managed Schema / Schemaless and SolrCloud concurrency issues

Key: SOLR-6137
URL: https://issues.apache.org/jira/browse/SOLR-6137
Project: Solr
Issue Type: Bug
Components: Schema and Analysis, SolrCloud
Reporter: Gregory Chanan

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6145) Concurrent Schema API field additions can result in endless loop


 [ 
https://issues.apache.org/jira/browse/SOLR-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-6145:
-

Attachment: concurrent_updates_and_schema_api.patch

[~alexey] gave me the attached patch, to {{TestCloudManagedSchemaAddField}}, 
that triggers the problem.

 Concurrent Schema API field additions can result in endless loop
 

 Key: SOLR-6145
 URL: https://issues.apache.org/jira/browse/SOLR-6145
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis
Reporter: Steve Rowe
Assignee: Steve Rowe
Priority: Critical
 Attachments: concurrent_updates_and_schema_api.patch


 The optimistic concurrency loop in {{ManagedIndexSchema.addFields()}} is the 
 likely culprit.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6137) Managed Schema / Schemaless and SolrCloud concurrency issues

2014-06-05 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-6137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019456#comment-14019456
]

Yonik Seeley commented on SOLR-6137:

bq. One way to fix this may be to return the schema ZK node version from Schema
API modification methods, and add a request param requiring a minimum schema ZK
node version prior to binding a schema snapshot to the request.

That's an interesting idea, but it would then require users of the schema API
to add versions to all their subsequent update requests (and no clear idea
when they could stop, and it gets even more interestnig with multiple update
clients).
An alternative would be to proactively check if there is a new schema version
if we fail to find a field. The overhead shouldn't be an issue if we would
otherwise fail the request.

We should be careful to only do this check when necessary. For example, it
seems like that queries for non-existent fields should not normally trigger a
ZK lookup (and the schema only changes with the IndexSearcher anyway). But
then again, a delete by query should check (it's indexing side, and one would
want to be able to add a field, add a doc with that field, then do a DBQ with
that field). It's context specific - tricky stuff.

Managed Schema / Schemaless and SolrCloud concurrency issues

Key: SOLR-6137
URL: https://issues.apache.org/jira/browse/SOLR-6137
Project: Solr
Issue Type: Bug
Components: Schema and Analysis, SolrCloud
Reporter: Gregory Chanan

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6147) QUERYHANDLER in Solr's admin GUI should be REQUESTHANDLER

2014-06-05 Thread David Smiley (JIRA)

David Smiley created SOLR-6147:
--

 Summary: QUERYHANDLER in Solr's admin GUI should be REQUESTHANDLER
 Key: SOLR-6147
 URL: https://issues.apache.org/jira/browse/SOLR-6147
 Project: Solr
  Issue Type: Improvement
  Components: web gui
Reporter: David Smiley
Priority: Minor


In the admin UI, go to Plugins / Stats where you'll see a QUERYHANDLER 
section.  That should be called REQUESTHANDLER, and likewise the URL to it 
should match.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (SOLR-6138) Solr core load limit


 [ 
https://issues.apache.org/jira/browse/SOLR-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

HuangTongwen reopened SOLR-6138:



I am sorry for the first time I decribe that problem is not complete.
At this experiment,we  have adjusted the max files numbers that linux can 
open.we set it 165556 it is very big than the default number.At the same time, 
we also have changed the znode size limit.We need know that why exception 
happened like that. Sometime I suspect that it is a problem in solr.



 Solr core load limit
 

 Key: SOLR-6138
 URL: https://issues.apache.org/jira/browse/SOLR-6138
 Project: Solr
  Issue Type: Bug
  Components: clients - java
Affects Versions: 4.6, 4.7.1
 Environment: ubuntu 12.04
 memery 20G
Reporter: HuangTongwen
  Labels: test
   Original Estimate: 840h
  Remaining Estimate: 840h

 We want to enrich our search ability by solr.We do an exercise for test that 
 how many cores in one machine solr cores can support.
 We find we can create more than 2000 cores without datas in one machine.But 
 when we create cores with data ,we just can create about 1000 cores,after 
 more t han 1000 cores,we meet many errors like following I will apend it .If 
 you have meets the same or similar problem,please tell me.
 I would be grateful if you could help me.
 Hear are some errors:
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:46:15  ERROR   ShardLeaderElectionContext  There was a problem 
 trying to register as the 
 leader:org.apache.zookeeper.KeeperException$NodeExistsException: 
 KeeperErrorCode = NodeExists for /collections/ctest.test.3521/leaders/shard1
 09:46:15  WARNElectionContext cancelElection did not find 
 election node to remove
 09:46:16  WARNRecoveryStrategyStopping recovery for 
 zkNodeName=core_node1core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyError while trying to recover. 
 core=ctest.test.3521:org.apache.solr.common.SolrException: No registered 
 leader was found, collection:ctest.test.3521 slice:shard1
 09:46:17  ERROR   RecoveryStrategyRecovery failed - trying 
 again... (0) core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyRecovery failed - interrupted. 
 core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyRecovery failed - I give up. 
 core=ctest.test.3521
 09:46:18  WARNRecoveryStrategyStopping recovery for

[jira] [Updated] (SOLR-6138) Solr core load exception


 [ 
https://issues.apache.org/jira/browse/SOLR-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

HuangTongwen updated SOLR-6138:
---

Summary: Solr core load exception  (was: Solr core load limit)

 Solr core load exception
 

 Key: SOLR-6138
 URL: https://issues.apache.org/jira/browse/SOLR-6138
 Project: Solr
  Issue Type: Bug
  Components: clients - java
Affects Versions: 4.6, 4.7.1
 Environment: ubuntu 12.04
 memery 20G
Reporter: HuangTongwen
  Labels: test
   Original Estimate: 840h
  Remaining Estimate: 840h

 We want to enrich our search ability by solr.We do an exercise for test that 
 how many cores in one machine solr cores can support.
 We find we can create more than 2000 cores without datas in one machine.But 
 when we create cores with data ,we just can create about 1000 cores,after 
 more t han 1000 cores,we meet many errors like following I will apend it .If 
 you have meets the same or similar problem,please tell me.
 I would be grateful if you could help me.
 Hear are some errors:
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:43:29  WARNSolrResourceLoader  Can't find (or read) directory 
 to add to classloader: /non/existent/dir/yields/warning (resolved as: 
 /non/existent/dir/yields/warning).
 09:46:15  ERROR   ShardLeaderElectionContext  There was a problem 
 trying to register as the 
 leader:org.apache.zookeeper.KeeperException$NodeExistsException: 
 KeeperErrorCode = NodeExists for /collections/ctest.test.3521/leaders/shard1
 09:46:15  WARNElectionContext cancelElection did not find 
 election node to remove
 09:46:16  WARNRecoveryStrategyStopping recovery for 
 zkNodeName=core_node1core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyError while trying to recover. 
 core=ctest.test.3521:org.apache.solr.common.SolrException: No registered 
 leader was found, collection:ctest.test.3521 slice:shard1
 09:46:17  ERROR   RecoveryStrategyRecovery failed - trying 
 again... (0) core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyRecovery failed - interrupted. 
 core=ctest.test.3521
 09:46:17  ERROR   RecoveryStrategyRecovery failed - I give up. 
 core=ctest.test.3521
 09:46:18  WARNRecoveryStrategyStopping recovery for 
 zkNodeName=core_node1core=ctest.test.3521
 10:01:58  ERROR   SolrCoreorg.apache.solr.common.SolrException: 
 Error handling 'status' action
 10:01:58  ERROR   SolrDispatchFilter  
 null:org.apache.solr.common.SolrException: Error handling 'status' action
 10:15:59  ERROR   ZkController

[jira] [Updated] (SOLR-6138) Solr ERROR RecoveryStrategy Recovery failed