[jira] [Commented] (SOLR-9317) ADDREPLICA command should be more flexible and add 'n' replicas to a collection,shard

[Nitin Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387168#comment-15387168
 ] 

Nitin Sharma commented on SOLR-9317:


This is a bit vague. What does automatically identify mean? Will it be based on 
nodes with least cores or nodes that do not have a replica for a given 
collection. If we take it a step further it could also be based on nodes that 
have the least disk usage. What was in your mind?

> ADDREPLICA command should be more flexible and add 'n' replicas to a 
> collection,shard
> -
>
> Key: SOLR-9317
> URL: https://issues.apache.org/jira/browse/SOLR-9317
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
>
> It should automatically identify the nodes where these replicas should be 
> created as well



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7036) Faster method for group.facet

[Jamie Swain (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387126#comment-15387126
 ] 

Jamie Swain commented on SOLR-7036:
---

[~dannytei1]  Hi, thanks for your response.  I took off DocValues, and with a 
quick local test it seems to be working very well. :-)  I'm going to set up a 
full-scale comparison later tonight or tomorrow and I'll post back with 
results.  

> Faster method for group.facet
> -
>
> Key: SOLR-7036
> URL: https://issues.apache.org/jira/browse/SOLR-7036
> Project: Solr
>  Issue Type: Improvement
>  Components: faceting
>Affects Versions: 4.10.3
>Reporter: Jim Musil
>Assignee: Erick Erickson
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-7036.patch, SOLR-7036.patch, SOLR-7036.patch, 
> SOLR-7036.patch, jstack-output.txt, performance.txt, source_for_patch.zip
>
>
> This is a patch that speeds up the performance of requests made with 
> group.facet=true. The original code that collects and counts unique facet 
> values for each group does not use the same improved field cache methods that 
> have been added for normal faceting in recent versions.
> Specifically, this approach leverages the UninvertedField class which 
> provides a much faster way to look up docs that contain a term. I've also 
> added a simple grouping map so that when a term is found for a doc, it can 
> quickly look up the group to which it belongs.
> Group faceting was very slow for our data set and when the number of docs or 
> terms was high, the latency spiked to multiple second requests. This solution 
> provides better overall performance -- from an average of 54ms to 32ms. It 
> also dropped our slowest performing queries way down -- from 6012ms to 991ms.
> I also added a few tests.
> I added an additional parameter so that you can choose to use this method or 
> the original. Add group.facet.method=fc to use the improved method or 
> group.facet.method=original which is the default if not specified.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7388) Add IntRangeField, FloatRangeField, LongRangeField

[Nicholas Knize (JIRA)]

Nicholas Knize created LUCENE-7388:
--

 Summary: Add IntRangeField, FloatRangeField, LongRangeField
 Key: LUCENE-7388
 URL: https://issues.apache.org/jira/browse/LUCENE-7388
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Nicholas Knize


This is the follow on to LUCENE-7381 for adding support for indexing and 
querying on {{int}}, {{float}}, and {{long}} ranges.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7381) Add new RangeField

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize resolved LUCENE-7381.

Resolution: Done

> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7381) Add new RangeField

[Nicholas Knize (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387072#comment-15387072
 ] 

Nicholas Knize commented on LUCENE-7381:


+1 to iterate in sandbox. I'll also add {{IntRangeField}}, 
{{DoubleRangeField}}, and {{LongRangeField}} in a separate issue. And 
{{GeoBBoxField}} also in another issue.

> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7381) Add new RangeField

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387069#comment-15387069
 ] 

ASF subversion and git services commented on LUCENE-7381:
-

Commit 7dfcfcc71851abe114d5db716bc5d099b540e3fe in lucene-solr's branch 
refs/heads/branch_6x from [~nknize]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7dfcfcc ]

LUCENE-7381: add point based DoubleRangeField and RangeFieldQuery for indexing 
and querying on Ranges up to 4 dimensions


> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7381) Add new RangeField

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387049#comment-15387049
 ] 

ASF subversion and git services commented on LUCENE-7381:
-

Commit f5964ec6e45adc8fc7644bbe60d9e5966bf11ab7 in lucene-solr's branch 
refs/heads/master from [~nknize]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f5964ec ]

LUCENE-7381: add point based DoubleRangeField and RangeFieldQuery for indexing 
and querying on Ranges up to 4 dimensions


> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (64bit/jdk1.8.0_92) - Build # 333 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/333/
Java: 64bit/jdk1.8.0_92 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestManagedSchemaAPI

Error Message:
ObjectTracker found 8 object(s) that were not released!!! 
[MDCAwareThreadPoolExecutor, TransactionLog, TransactionLog, 
MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, 
MDCAwareThreadPoolExecutor, MockDirectoryWrapper]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 8 object(s) that were not 
released!!! [MDCAwareThreadPoolExecutor, TransactionLog, TransactionLog, 
MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, 
MDCAwareThreadPoolExecutor, MockDirectoryWrapper]
at __randomizedtesting.SeedInfo.seed([42339858A39A8FDA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestManagedSchemaAPI

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.schema.TestManagedSchemaAPI_42339858A39A8FDA-001\tempDir-001\node2\testschemaapi_shard1_replica2\data\tlog\tlog.001:
 java.nio.file.FileSystemException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.schema.TestManagedSchemaAPI_42339858A39A8FDA-001\tempDir-001\node2\testschemaapi_shard1_replica2\data\tlog\tlog.001:
 The process cannot access the file because it is being used by another 
process. 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.schema.TestManagedSchemaAPI_42339858A39A8FDA-001\tempDir-001\node2\testschemaapi_shard1_replica2\data\tlog:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.schema.TestManagedSchemaAPI_42339858A39A8FDA-001\tempDir-001\node2\testschemaapi_shard1_replica2\data\tlog

C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.schema.TestManagedSchemaAPI_42339858A39A8FDA-001\tempDir-001\node2\testschemaapi_shard1_replica2\data:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.schema.TestManagedSchemaAPI_42339858A39A8FDA-001\tempDir-001\node2\testschemaapi_shard1_replica2\data

[jira] [Commented] (SOLR-8715) New Admin UI's Schema screen fails for some fields

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386914#comment-15386914
 ] 

Alexandre Rafalovitch commented on SOLR-8715:
-

5.5.3 might be coming out soon. Any chance we can fold this into that? We 
already have the solution tested and it is super trivial. Yet, it stops the new 
Admin UI being usable in fairly common situations.

> New Admin UI's Schema screen fails for some fields
> --
>
> Key: SOLR-8715
> URL: https://issues.apache.org/jira/browse/SOLR-8715
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 5.5, 6.0
> Environment: mac, firefox
>Reporter: Alexandre Rafalovitch
>Assignee: Upayavira
>  Labels: admin-interface
> Attachments: Problem shown in the released 5.5 version.png
>
>
> In techproducts example, using new Admin UI and trying to load the Schema for 
> text field causes blank screen and the Javascript error in the developer 
> console:
> {noformat}
> Error: row.flags is undefined
> getFieldProperties@http://localhost:8983/solr/js/angular/controllers/schema.js:482:40
> $scope.refresh/http://localhost:8983/solr/js/angular/controllers/schema.js:76:38
> 
> {noformat}
> Tested with 5.5rc3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_92) - Build # 17315 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17315/
Java: 64bit/jdk1.8.0_92 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.overseer.ZkStateWriterTest

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.overseer.ZkStateWriterTest: 1) Thread[id=378, 
name=watches-52-thread-1, state=TIMED_WAITING, group=TGRP-ZkStateWriterTest]
 at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.overseer.ZkStateWriterTest: 
   1) Thread[id=378, name=watches-52-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([8E642426B76959AA]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.overseer.ZkStateWriterTest

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=378, name=watches-52-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest] at sun.misc.Unsafe.park(Native Method)
 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)  
   at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=378, name=watches-52-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([8E642426B76959AA]:0)




Build Log:
[...truncated 10702 lines...]
   [junit4] Suite: org.apache.solr.cloud.overseer.ZkStateWriterTest
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.overseer.ZkStateWriterTest_8E642426B76959AA-001/init-core-data-001
   [junit4]   2> 62878 INFO  
(SUITE-ZkStateWriterTest-seed#[8E642426B76959AA]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 62881 INFO  
(TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[8E642426B76959AA]) [ 
   ] o.a.s.SolrTestCaseJ4 ###Starting 

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1077 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1077/

13 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrReplicationDistributedZkTest

Error Message:
ObjectTracker found 48 object(s) that were not released!!! [InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 48 object(s) that were not 
released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient]
at __randomizedtesting.SeedInfo.seed([DC38D6D73F50B7BE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrReplicationDistributedZkTest

Error Message:

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+127) - Build # 1220 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1220/
Java: 32bit/jdk-9-ea+127 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:41316/_/db/forceleader_test_collection_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:41316/_/db/forceleader_test_collection_shard1_replica2]
at 
__randomizedtesting.SeedInfo.seed([54FC5D2483DB0D7D:B26B69E4BA59F41C]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:739)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use

[Hrishikesh Gadre (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386814#comment-15386814
 ] 

Hrishikesh Gadre commented on SOLR-9242:


Found one more unit test failure on Windows. Investigating...

http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/5996/

> Collection level backup/restore should provide a param for specifying the 
> repository implementation it should use
> -
>
> Key: SOLR-9242
> URL: https://issues.apache.org/jira/browse/SOLR-9242
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hrishikesh Gadre
>Assignee: Varun Thacker
> Fix For: 6.2
>
> Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, 
> SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch
>
>
> SOLR-7374 provides BackupRepository interface to enable storing Solr index 
> data to a configured file-system (e.g. HDFS, local file-system etc.). This 
> JIRA is to track the work required to extend this functionality at the 
> collection level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_92) - Build # 5996 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/5996/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
Error from server at https://127.0.0.1:64240/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_820C98D83C85E25C-001/tempDir-002/mytestbackup/

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:64240/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_820C98D83C85E25C-001/tempDir-002/mytestbackup/
at 
__randomizedtesting.SeedInfo.seed([820C98D83C85E25C:A58A70292798FA4]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:366)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1270)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:206)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[jira] [Commented] (LUCENE-7381) Add new RangeField

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386801#comment-15386801
 ] 

Adrien Grand commented on LUCENE-7381:
--

+1

I suspect we can make things faster by reorganizing how the visitor computes 
the relation between the cell and the range but this is sandbox so maybe we 
should get this in and iterate from there?

> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9038) Support snapshot management functionality for a solr collection

[Hrishikesh Gadre (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre updated SOLR-9038:
---
Summary: Support snapshot management functionality for a solr collection  
(was: Ability to create/delete/list snapshots for a solr collection)

> Support snapshot management functionality for a solr collection
> ---
>
> Key: SOLR-9038
> URL: https://issues.apache.org/jira/browse/SOLR-9038
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
>
> Currently work is under-way to implement backup/restore API for Solr cloud 
> (SOLR-5750). SOLR-5750 is about providing an ability to "copy" index files 
> and collection metadata to a configurable location. 
> In addition to this, we should also provide a facility to create "named" 
> snapshots for Solr collection. Here by "snapshot" I mean configuring the 
> underlying Lucene IndexDeletionPolicy to not delete a specific commit point 
> (e.g. using PersistentSnapshotIndexDeletionPolicy). This should not be 
> confused with SOLR-5340 which implements core level "backup" functionality.
> The primary motivation of this feature is to decouple recording/preserving a 
> known consistent state of a collection from actually "copying" the relevant 
> files to a physically separate location. This decoupling have number of 
> advantages
> - We can use specialized data-copying tools for transferring Solr index 
> files. e.g. in Hadoop environment, typically 
> [distcp|https://hadoop.apache.org/docs/r1.2.1/distcp2.html] tool is used to 
> copy files from one location to other. This tool provides various options to 
> configure degree of parallelism, bandwidth usage as well as integration with 
> different types and versions of file systems (e.g. AWS S3, Azure Blob store 
> etc.)
> - This separation of concern would also help Solr to focus on the key 
> functionality (i.e. querying and indexing) while delegating the copy 
> operation to the tools built for that purpose.
> - Users can decide if/when to copy the data files as against creating a 
> snapshot. e.g. a user may want to create a snapshot of a collection before 
> making an experimental change (e.g. updating/deleting docs, schema change 
> etc.). If the experiment is successful, he can delete the snapshot (without 
> having to copy the files). If the experiment is failed, then he can copy the 
> files associated with the snapshot and restore.
> Note that Apache Blur project is also providing a similar feature 
> [BLUR-132|https://issues.apache.org/jira/browse/BLUR-132]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9326) Ability to create/delete/list snapshots for a solr collection

[Hrishikesh Gadre (JIRA)]

Hrishikesh Gadre created SOLR-9326:
--

 Summary: Ability to create/delete/list snapshots for a solr 
collection
 Key: SOLR-9326
 URL: https://issues.apache.org/jira/browse/SOLR-9326
 Project: Solr
  Issue Type: Sub-task
Reporter: Hrishikesh Gadre






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core

[Hrishikesh Gadre (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386789#comment-15386789
 ] 

Hrishikesh Gadre commented on SOLR-9269:


[~dsmiley] I have created a pull request. It addresses all the review comments. 
All the tests as well as precommit is passing for me.

Thses are your latest comments 

Ref: 
https://github.com/hgadre/lucene-solr/commit/1ab2b5022a2ed970e0bad733a4bdb284bb7a0830#commitcomment-18106713
Ref: 
https://github.com/hgadre/lucene-solr/commit/1ab2b5022a2ed970e0bad733a4bdb284bb7a0830#commitcomment-18106676

bq. I haven't dealt with this class much before; lots of complicated code here. 
The code you added seems to be only for a full copy replication, and it appears 
that is to a new index dir, hence the existing logic to remove an "old" index 
dir. In this circumstance why delete only the non-snapshot index files; 
shouldn't we basically remove everything and clear all snapshots? Or, and this 
may be complicated, try and retain the snapshots by copying the snapshotted 
files from old to new?

bq. This is the only spot that calls deleteSnapshotIndexFiles, and I'm not 
completely clear as to its comments above. How would the indexDirPath not be 
the core index dir? Or in other words, why does the snapshot commit metadata 
have the index dir String when it might be implied as the same as for the Solr 
core?

There are some operations (e.g. restore or full replication) where Solr changes 
the *current* index directory. When the old index directory before the 
operation has snapshots stored, we have two options,
(a) Copy snapshot files to the new directory.
(b) Keep the old directory (and the relevant index files) around until the 
snapshot exist. When the snapshot is deleted, cleanup the index files. Note 
that this cleanup is performed *only* when the directory containing snapshot 
files != the *current* index directory. When directory containing snapshot 
files == the *current* index directory, we delegate file management to 
IndexDeletionPolicy implementation.

Option (a) is not workable since
 - There can be a file naming conflict between files corresponding to index 
commit referred by the snapshot and the index commit copied from external 
source (e.g. from external backup or some other Solr server).
 - Copying files would require more disk I/O which is not really necessary from 
Solr perspective (since the snapshot files are just a corpus intended for 
backup/restore use-case).

Hence I implemented option (b) in this patch.

bq. BTW if the path does need to continue to be a part of the snapshot 
metadata, then that draws suspicion to the isSnapshotted(long) method  
signature since it ought to also have the path? Or maybe snapshot metadata 
management needs to be fundamentally managed per index dir path?

Yes good catch! I fixed this problem. I also addressed the case of Lucene 
segment sharing between multiple index commits as well as multiple snapshots 
referring to the same index commit by implementing a reference counting 
mechanism. I have added unit tests to verify this behavior as well.


> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level. 
> Please refer to parent JIRA for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7495) Unexpected docvalues type NUMERIC when grouping by a int facet

[Nick Coult (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386783#comment-15386783
 ] 

Nick Coult commented on SOLR-7495:
--

The patch did not cleanly apply but it does compile and run correctly, thanks!

> Unexpected docvalues type NUMERIC when grouping by a int facet
> --
>
> Key: SOLR-7495
> URL: https://issues.apache.org/jira/browse/SOLR-7495
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.3
>Reporter: Fabio Batista da Silva
> Attachments: SOLR-7495.patch, SOLR-7495.patch
>
>
> Hey All,
> After upgrading from solr 4.10 to 5.1 with solr could
> I'm getting a IllegalStateException when i try to facet a int field.
> IllegalStateException: unexpected docvalues type NUMERIC for field 'year' 
> (expected=SORTED). Use UninvertingReader or index with docvalues.
> schema.xml
> {code}
> 
> 
> 
> 
> 
> 
>  multiValued="false" required="true"/>
>  multiValued="false" required="true"/>
> 
> 
>  stored="true"/>
> 
> 
> 
>  />
>  sortMissingLast="true"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  precisionStep="0" positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="100">
> 
> 
>  words="stopwords.txt" />
> 
>  maxGramSize="15"/>
> 
> 
> 
>  words="stopwords.txt" />
>  synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
> 
> 
> 
>  positionIncrementGap="100">
> 
> 
>  words="stopwords.txt" />
> 
>  maxGramSize="15"/>
> 
> 
> 
>  words="stopwords.txt" />
>  synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
> 
> 
> 
>  class="solr.SpatialRecursivePrefixTreeFieldType" geo="true" 
> distErrPct="0.025" maxDistErr="0.09" units="degrees" />
> 
> id
> name
> 
> 
> {code}
> query :
> {code}
> http://solr.dev:8983/solr/my_collection/select?wt=json=id=index_type:foobar=true=year_make_model=true=true=year
> {code}
> Exception :
> {code}
> ull:org.apache.solr.common.SolrException: Exception during facet.field: year
> at org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:627)
> at org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:612)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:566)
> at 
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:637)
> at 
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:280)
> at 
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:106)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:222)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at 
> 

[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...

[hgadre]

GitHub user hgadre opened a pull request:

https://github.com/apache/lucene-solr/pull/52

[SOLR-9269] Ability to create/delete/list snapshots for a solr core

- Introduced admin operations to create/delete/list snapshots
  at the Solr core level
- Introduced Snapshot management functionality utilizing Lucene
  IndexDeletionPolicy. The management of data files in the *current*
  index directory is deleted to Solr IndexDeletionPolicy.
- As part of restore operation (or full replication), Solr changes the
  index directory. Since snapshots can be present in the old directory
  (before the operation), introduced logic to preserve those index files.
  When the snapshot is deleted, this logic deletes those files if they
  are not referred by some other snapshot.
- Unit tests to verify new functionlity.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hgadre/lucene-solr SOLR-9269_fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/52.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #52


commit 3483b61f7e1599087eef3976c2a98673fdd79282
Author: Hrishikesh Gadre 
Date:   2016-06-13T21:40:12Z

[SOLR-9269] Ability to create/delete/list snapshots for a solr core

- Introduced admin operations to create/delete/list snapshots
  at the Solr core level
- Introduced Snapshot management functionality utilizing Lucene
  IndexDeletionPolicy. The management of data files in the *current*
  index directory is deleted to Solr IndexDeletionPolicy.
- As part of restore operation (or full replication), Solr changes the
  index directory. Since snapshots can be present in the old directory
  (before the operation), introduced logic to preserve those index files.
  When the snapshot is deleted, this logic deletes those files if they
  are not referred by some other snapshot.
- Unit tests to verify new functionlity.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7381) Add new RangeField

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-7381:
---
Attachment: LUCENE-7381.patch

Attached wrong patch. Proper one now attached.

> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7381) Add new RangeField

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-7381:
---
Attachment: (was: LUCENE-7381.patch)

> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Release Solr 5.5.3

[Erick Erickson]

I also would like to get SOLR-7280 in, Noble and I just checked it in
to that branch as well.

Erick

On Wed, Jul 20, 2016 at 3:02 PM, Anshum Gupta  wrote:
> As Shai mentioned, it is actually about SSL + indexing requests leading to
> unstable state in Solr.
>
> How quickly that state is reached is a function of # indexing requests.
> Thanks for correcting me on that one :-).
>
> On Wed, Jul 20, 2016 at 1:35 PM, David Smiley 
> wrote:
>>
>> Okay.  BTW SOLR-9290 isn't "Just" high indexing rates, but it's for those
>> using SSL too -- correct me if I'm wrong.  We don't want to raise alarm
>> bells too loudly :-)
>>
>> On Wed, Jul 20, 2016 at 4:18 PM Anshum Gupta 
>> wrote:
>>>
>>> Hi,
>>>
>>> With SOLR-9290 fixed, I think it calls for a bug fix release as it
>>> impacts all users with high indexing rates.
>>>
>>> If someone else wants to work on the release, I am fine with it else,
>>> I'll be happy to be the RM and cut an RC a week from now.
>>>
>>> --
>>> Anshum Gupta
>>
>> --
>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>> http://www.solrenterprisesearchserver.com
>
>
>
>
> --
> Anshum Gupta

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7381) Add new RangeField

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-7381:
---
Attachment: LUCENE-7381.patch

Instead of having to create a new impl for each query I just simplified the 
{{RangeFieldComparator}}. Now its only responsibility is to provide the range 
relation logic. I then moved the relation handling up into the 
IntersectsVisitor. I think this is better and more properly organized and 
should be easier to follow.

> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386729#comment-15386729
 ] 

ASF subversion and git services commented on SOLR-7280:
---

Commit bcaf4999d83a5a4cbc7f02ecc8d1f00e4a4a7212 in lucene-solr's branch 
refs/heads/branch_5x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bcaf499 ]

SOLR-7280: Load cores in sorted order & limit threads to improve cluster 
stability
(cherry picked from commit 9fbb2fe)


> Load cores in sorted order and tweak coreLoadThread counts to improve cluster 
> stability on restarts
> ---
>
> Key: SOLR-7280
> URL: https://issues.apache.org/jira/browse/SOLR-7280
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
> Fix For: 6.2, 5.5.3
>
> Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, 
> SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch
>
>
> In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order 
> and tweaking some of the coreLoadThread counts, he was able to improve the 
> stability of a cluster with thousands of collections. We should explore some 
> of these changes and fold them into Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9314) Add all known transformers to TestRandomFlRTGCloud and fix any bugs found

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-9314:
---
Attachment: SOLR-9314.patch

WIP patch with new validators for {{\[shard\]}}, {{\[explain\]}}, 
{{\[value\]}},  {{\[geo\]}}, and {{\[subquery\]}}

The subquery validation tries to search for all docs with teh same field value 
as the current doc, asserting that there is always at least 1 match -- but this 
assertion currently fails ... by the looks of it this is (obviously) because it 
doesn't know to to use the realtime seracher re-opened by the RTG ... but based 
on how the SubQueryAugmenter is implemented, i'm not even certain how to go 
about it.

> Add all known transformers to TestRandomFlRTGCloud and fix any bugs found
> -
>
> Key: SOLR-9314
> URL: https://issues.apache.org/jira/browse/SOLR-9314
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-9314.patch
>
>
> SOLR-9285 is/has added a new TestRandomFlRTGCloud for testing out random 
> permutations of {{fl}} params in RTG requests.
> simple field names, field renames, field globs, and ValueSources/functions 
> are already tested -- adding additional DocTransformers is fairly straight 
> forward by implementing a simple FlValidator.
> This issue is being filed to track the work needed to:
> * review the list of all DocTransformers in solr and add test coverage of 
> them in this method
> * file any blocker issues if new bugs are found in this process
> * once all DocTransformers are accounted for, update TestRandomFlRTGCloud to 
> programatically compare the list of FlValidator's it knows about with the 
> list of all DocTransformers (similar to how 
> QueryEqualityTest.testParserCoverage works) to fail the test if anyone adds a 
> DocTransformer w/o adding corisponding test coverage to this class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Release Solr 5.5.3

[Anshum Gupta]

As Shai mentioned, it is actually about SSL + indexing requests leading to
unstable state in Solr.

How quickly that state is reached is a function of # indexing requests.
Thanks for correcting me on that one :-).

On Wed, Jul 20, 2016 at 1:35 PM, David Smiley 
wrote:

> Okay.  BTW SOLR-9290 isn't "Just" high indexing rates, but it's for those
> using SSL too -- correct me if I'm wrong.  We don't want to raise alarm
> bells too loudly :-)
>
> On Wed, Jul 20, 2016 at 4:18 PM Anshum Gupta 
> wrote:
>
>> Hi,
>>
>> With SOLR-9290 fixed, I think it calls for a bug fix release as it
>> impacts all users with high indexing rates.
>>
>> If someone else wants to work on the release, I am fine with it else,
>> I'll be happy to be the RM and cut an RC a week from now.
>>
>> --
>> Anshum Gupta
>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
>



-- 
Anshum Gupta

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 279 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/279/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
ObjectTracker found 4 object(s) that were not released!!! [SolrCore, 
MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not 
released!!! [SolrCore, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, 
MockDirectoryWrapper]
at __randomizedtesting.SeedInfo.seed([54ED3A8D8974495F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 1) 
Thread[id=37933, name=searcherExecutor-7780-thread-1, state=WAITING, 
group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
 at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.core.TestLazyCores: 
   1) Thread[id=37933, name=searcherExecutor-7780-thread-1, state=WAITING, 
group=TGRP-TestLazyCores]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at 

[jira] [Commented] (SOLR-9323) Add a new method getCollectionNamesFast() to ClusterSate which returns unverified list of collection names

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386660#comment-15386660
 ] 

Scott Blum commented on SOLR-9323:
--

+1 or expose `Map getCollectionStates()`, may as 
getCollectionRefMap().  I was worried getCollectionsMap() would be insufficient.

> Add a new method getCollectionNamesFast() to ClusterSate which returns 
> unverified list of collection names
> --
>
> Key: SOLR-9323
> URL: https://issues.apache.org/jira/browse/SOLR-9323
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>
> Currently the {{getCollections()}} method is deprecated and everyone is 
> forced to use {{getCollectionsMap()}} which could be extremely expensive 
> depending on the no:of collections. The {{getCollectionsMap()}} method is 
> very expensive and should never be invoked on each request



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Release Solr 5.5.3

[Shai Erera]

Right, it affects SSL-enabled Solr more quickly. I do not know if it
eventually affects non-SSL Solr, I guess not ...

Also, it's not about high indexing rates, as it's reproduced with 9
docs/sec, IIRC.

Anyway, it certainly affected several users already, so I'm +1 for the
release.

Shai

On Wed, Jul 20, 2016 at 11:35 PM David Smiley 
wrote:

> Okay.  BTW SOLR-9290 isn't "Just" high indexing rates, but it's for those
> using SSL too -- correct me if I'm wrong.  We don't want to raise alarm
> bells too loudly :-)
>
> On Wed, Jul 20, 2016 at 4:18 PM Anshum Gupta 
> wrote:
>
>> Hi,
>>
>> With SOLR-9290 fixed, I think it calls for a bug fix release as it
>> impacts all users with high indexing rates.
>>
>> If someone else wants to work on the release, I am fine with it else,
>> I'll be happy to be the RM and cut an RC a week from now.
>>
>> --
>> Anshum Gupta
>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
>

Re: Release Solr 5.5.3

[David Smiley]

Okay.  BTW SOLR-9290 isn't "Just" high indexing rates, but it's for those
using SSL too -- correct me if I'm wrong.  We don't want to raise alarm
bells too loudly :-)

On Wed, Jul 20, 2016 at 4:18 PM Anshum Gupta  wrote:

> Hi,
>
> With SOLR-9290 fixed, I think it calls for a bug fix release as it impacts
> all users with high indexing rates.
>
> If someone else wants to work on the release, I am fine with it else, I'll
> be happy to be the RM and cut an RC a week from now.
>
> --
> Anshum Gupta
>
-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

Release Solr 5.5.3

[Anshum Gupta]

Hi,

With SOLR-9290 fixed, I think it calls for a bug fix release as it impacts
all users with high indexing rates.

If someone else wants to work on the release, I am fine with it else, I'll
be happy to be the RM and cut an RC a week from now.

-- 
Anshum Gupta

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_92) - Build # 17313 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17313/
Java: 64bit/jdk1.8.0_92 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch

Error Message:
CollectionStateWatcher wasn't cleared after completion

Stack Trace:
java.lang.AssertionError: CollectionStateWatcher wasn't cleared after completion
at 
__randomizedtesting.SeedInfo.seed([C4D060B161692DC5:99EBAFC12664B2FB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13078 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating dataDir: 

[JENKINS] Lucene-Solr-6.x-MacOSX (64bit/jdk1.8.0) - Build # 292 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/292/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[http://127.0.0.1:64649/sf/forceleader_test_collection_shard1_replica1]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[http://127.0.0.1:64649/sf/forceleader_test_collection_shard1_replica1]
at 
__randomizedtesting.SeedInfo.seed([5D7CEDB391755B8D:BBEBD973A8F7A2EC]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:753)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
at 
org.apache.solr.cloud.ForceLeaderTest.assertSendDocFails(ForceLeaderTest.java:315)
at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_92) - Build # 1218 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1218/
Java: 64bit/jdk1.8.0_92 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val modified' for path 'response/params/y/c' 
full output: {   "responseHeader":{ "status":0, "QTime":0},   
"response":{ "znodeVersion":0, "params":{"x":{ "a":"A val", 
"b":"B val", "":{"v":0},  from server:  
https://127.0.0.1:38411/cunnk/uj/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val modified' for 
path 'response/params/y/c' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "response":{
"znodeVersion":0,
"params":{"x":{
"a":"A val",
"b":"B val",
"":{"v":0},  from server:  
https://127.0.0.1:38411/cunnk/uj/collection1
at 
__randomizedtesting.SeedInfo.seed([CF7D9B18BEC16D65:4729A4C2103D009D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:194)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386382#comment-15386382
 ] 

Joel Bernstein commented on SOLR-9252:
--

Yes the model can be saved as a document. The model contains the features that 
were used to create the model. And the associated weights for each feature.

feature selection can be done as a separate step and stored in the index. 
Feature selection takes time and it's likely users will want to view the 
features that were extracted from the training data. Also, features could be 
used for other purposes as they are really just a list of terms that provide 
the most "information" about a training set. So it would be useful to store 
them.

The training function reads the features as a stream, so they can either be a 
stored feature set, or generated on the fly.




> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5944) Support updates of numeric DocValues

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-5944:
---
Attachment: SOLR-5944.patch

Added tests for the two scenarios described in the previous comment.
# TestInPlaceUpdatesDistrib.outOfOrderDeleteUpdatesIndividualReplicaTest() for 
the scenario where in-place update arrives after a delete.
# Added another scenario to 
TestInPlaceUpdatesDistrib.delayedReorderingFetchesMissingUpdateFromLeaderTest() 
where a missing update needs to be fetched from leader, even though the 
document itself has been deleted at the leader.

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9325) solr.log written to {solrRoot}/server/logs instead of location specified by SOLR_LOGS_DIR

[Tim Parker (JIRA)]

Tim Parker created SOLR-9325:


 Summary: solr.log written to {solrRoot}/server/logs instead of 
location specified by SOLR_LOGS_DIR
 Key: SOLR-9325
 URL: https://issues.apache.org/jira/browse/SOLR-9325
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.0.1, 5.5.2
 Environment: 64-bit CentOS 7 with latest patches, JVM 1.8.0.92
Reporter: Tim Parker


(6.1 is probably also affected, but we've been blocked by SOLR-9231)

solr.log should be written to the directory specified by the SOLR_LOGS_DIR 
environment variable, but instead it's written to {solrRoot}/server/logs.

This results in requiring that solr is installed on a writable device, which 
leads to two problems:
1) solr installation can't live on a shared device (single copy shared by two 
or more VMs)
2) solr installation is more difficult to lock down

Solr should be able to run without error in this test scenario:

burn the Solr directory tree onto a CD-ROM
Mount this CD as /solr
run Solr from there (with appropriate environment variables set, of course)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386322#comment-15386322
 ] 

David Smiley commented on SOLR-9252:


So is a "model" ultimately a Document?  I'm guessing so since you mentioned 
putting them in Solr and having millions of them.

Are there intermediate steps where data is put into Solr that isn't a model?  
You mentioned feature selection but it's not clear if that is materialized into 
Solr data or if it's used purely in-memory transiently.

Thanks for your explanation.

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386257#comment-15386257
 ] 

Joel Bernstein commented on SOLR-9252:
--

Here is an example of how you could use the models in a search context.

1) Identify a training set for each user based on usage logs. This could 
involve what the user has viewed before, or liked. To increase the size of the 
training set you could use graph queries to find documents that co-occur in the 
same session most frequently with documents that user has viewed or liked.

2) Optimize a model for the specific training set and store the model in a 
solrcloud collection. 

3) Use the model either in the re-ranker to boost documents based on the score 
from the model, or as part of an alerting engine to push documents the user 
might be interested in.

4) Backgrounds daemons could run in Solr that would build models for users. 
This would result in possibly millions of models, which is fine, because the 
models are simply stored in a SolrCloud collection.



> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_92) - Build # 17312 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17312/
Java: 32bit/jdk1.8.0_92 -server -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.cloud.RecoveryZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12928, 
name=updateExecutor-2319-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=12928, name=updateExecutor-2319-thread-1, 
state=RUNNABLE, group=TGRP-RecoveryZkTest]
at 
__randomizedtesting.SeedInfo.seed([235E3739C6FE8A0F:AB0A08E36802E7F7]:0)
Caused by: org.apache.solr.common.SolrException: Replica: 
https://127.0.0.1:39227/ot/collection1/ should have been marked under leader 
initiated recovery in ZkController but wasn't.
at __randomizedtesting.SeedInfo.seed([235E3739C6FE8A0F]:0)
at 
org.apache.solr.cloud.LeaderInitiatedRecoveryThread.run(LeaderInitiatedRecoveryThread.java:88)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.update.AutoCommitTest.testMaxTime

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([235E3739C6FE8A0F:B9AA4ADB58641633]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:782)
at 
org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:243)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Created] (SOLR-9324) Support Secure Impersonation / Proxy User for solr authentication

[Gregory Chanan (JIRA)]

Gregory Chanan created SOLR-9324:


 Summary: Support Secure Impersonation / Proxy User for solr 
authentication
 Key: SOLR-9324
 URL: https://issues.apache.org/jira/browse/SOLR-9324
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrCloud
Reporter: Gregory Chanan


Solr should support Proxy User / Secure Impersonation for authentication, as 
supported by hadoop 
(http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Superusers.html)
 and supported by the hadoop AuthenticationFilter (which we use for the 
KerberosPlugin).

There are a number of use cases, but a common one is this:
There is a front end for searches (say, Hue http://gethue.com/) that supports 
its own login mechanisms.  If the cluster uses kerberos for authentication, hue 
must have kerberos credentials for each user, which is a pain to manage.  
Instead, hue can be allowed to impersonate known users from known machines so 
it only needs its own kerberos credentials.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7315) Flexible "standard" query parser parses on whitespace

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated LUCENE-7315:
---
Attachment: LUCENE-7315.patch

WIP patch against master, generated files not included ({{ant javacc-flexible}} 
in {{lucene/queryparser/}} will generate them), still has nocommits and failing 
tests.

In addition to enabling not splitting on whitespace prior to text analysis, the 
patch includes the following changes:

* Changed {{TermQueryNode}}'s {{positionIncrement}} name to {{position}}, since 
that's what it really holds.
* {{SynonymQueryNode}}/{{Builder}} now produces a {{SynonymQuery}} instead of a 
boolean query.
* Refactored {{AnalyzerQueryNodeProcessor.postProcessNode()}} into shorter 
methods and made it simpler and easier to follow.
* Moved split-on-whitespace tests to the shared {{QueryParserTestBase}}.

Some challenges remain:

* Unlike the classic QP, the flexible standard QP appears to remove a top-level 
MUST boolean query, e.g. {{+(word)}} -> {{word}}.  Some of the 
split-on-whitespace shared tests will need to be specialized for each parser.
* There's no simple way to collapse the children of the boolean query produced 
for text containing whitespace when not splitting on whitespace into their 
ancestor boolean query (if there is one), so some of the shared 
split-on-whitespace tests are failing.
** The patch includes a {{FlattenQueryNodeProcessor}} meant to address this 
issue, but it's not working and I haven't figured out why yet.
* Recent master-only changes will likely make the branch_6x backport 
non-trivial, e.g LUCENE-7347. 

> Flexible "standard" query parser parses on whitespace
> -
>
> Key: LUCENE-7315
> URL: https://issues.apache.org/jira/browse/LUCENE-7315
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Attachments: LUCENE-7315.patch
>
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but in many cases they can't. Instead, preferably the queryparser 
> would parse around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8589) Add aliases to the LIST action results in the Collections API

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated SOLR-8589:

Attachment: SOLR-8589.patch

> Add aliases to the LIST action results in the Collections API
> -
>
> Key: SOLR-8589
> URL: https://issues.apache.org/jira/browse/SOLR-8589
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-8589.patch, SOLR-8589.patch, SOLR-8589.patch, 
> solr-8589-new-list-details-aliases.png
>
>
> Although it is possible to get a list of SolrCloud aliases vi an HTTP API, it 
> is not available as a typical query response, I believe it is only available 
> via the http API for zookeeper.
> The results from the LIST action in the Collections API is well-situated to 
> handle this. The current results are contained in a "collections" node, we 
> can simply add an "aliases" node if there are any aliases defined.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8589) Add aliases to the LIST action results in the Collections API

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated SOLR-8589:

Attachment: (was: SOLR-8589.patch)

> Add aliases to the LIST action results in the Collections API
> -
>
> Key: SOLR-8589
> URL: https://issues.apache.org/jira/browse/SOLR-8589
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-8589.patch, SOLR-8589.patch, SOLR-8589.patch, 
> solr-8589-new-list-details-aliases.png
>
>
> Although it is possible to get a list of SolrCloud aliases vi an HTTP API, it 
> is not available as a typical query response, I believe it is only available 
> via the http API for zookeeper.
> The results from the LIST action in the Collections API is well-situated to 
> handle this. The current results are contained in a "collections" node, we 
> can simply add an "aliases" node if there are any aliases defined.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-7280:
-
Fix Version/s: 5.5.3

> Load cores in sorted order and tweak coreLoadThread counts to improve cluster 
> stability on restarts
> ---
>
> Key: SOLR-7280
> URL: https://issues.apache.org/jira/browse/SOLR-7280
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
> Fix For: 6.2, 5.5.3
>
> Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, 
> SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch
>
>
> In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order 
> and tweaking some of the coreLoadThread counts, he was able to improve the 
> stability of a cluster with thousands of collections. We should explore some 
> of these changes and fold them into Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8589) Add aliases to the LIST action results in the Collections API

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated SOLR-8589:

Attachment: SOLR-8589.patch

New patch that accounts for SOLR-8995

> Add aliases to the LIST action results in the Collections API
> -
>
> Key: SOLR-8589
> URL: https://issues.apache.org/jira/browse/SOLR-8589
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-8589.patch, SOLR-8589.patch, SOLR-8589.patch, 
> solr-8589-new-list-details-aliases.png
>
>
> Although it is possible to get a list of SolrCloud aliases vi an HTTP API, it 
> is not available as a typical query response, I believe it is only available 
> via the http API for zookeeper.
> The results from the LIST action in the Collections API is well-situated to 
> handle this. The current results are contained in a "collections" node, we 
> can simply add an "aliases" node if there are any aliases defined.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386201#comment-15386201
 ] 

ASF subversion and git services commented on SOLR-7280:
---

Commit 9fbb2fe752b5baa224c33e0fd441cfb9082dd102 in lucene-solr's branch 
refs/heads/branch_5_5 from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9fbb2fe ]

SOLR-7280: Load cores in sorted order & limit threads to improve cluster 
stability


> Load cores in sorted order and tweak coreLoadThread counts to improve cluster 
> stability on restarts
> ---
>
> Key: SOLR-7280
> URL: https://issues.apache.org/jira/browse/SOLR-7280
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
> Fix For: 6.2
>
> Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, 
> SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch
>
>
> In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order 
> and tweaking some of the coreLoadThread counts, he was able to improve the 
> stability of a cluster with thousands of collections. We should explore some 
> of these changes and fold them into Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9320) A REPLACENODE command to decommission an existing node with another new node

[Nitin Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386191#comment-15386191
 ] 

Nitin Sharma commented on SOLR-9320:


Thanks for clarifying. I am aligning towards option a) as well (for simplicity 
and performance) but wanted to confirm the semantics. 

> A REPLACENODE command to decommission an existing node with another new node
> 
>
> Key: SOLR-9320
> URL: https://issues.apache.org/jira/browse/SOLR-9320
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
>
> The command should accept a source node and target node. recreate the 
> replicas in source node in the target and do a DLETENODE of source node



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8995) Minimize code with Lambdas

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386153#comment-15386153
 ] 

ASF subversion and git services commented on SOLR-8995:
---

Commit fb4de6adb1bd8b7b97999a98d8464e33ef9e965e in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fb4de6a ]

SOLR-8995: Use lambda to simplify CollectionsHandler


> Minimize code with Lambdas
> --
>
> Key: SOLR-8995
> URL: https://issues.apache.org/jira/browse/SOLR-8995
> Project: Solr
>  Issue Type: Task
>Reporter: Noble Paul
>Assignee: Noble Paul
> Attachments: SOLR-8995.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 340 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/340/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestRequestForwarding

Error Message:
2 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestRequestForwarding: 1) Thread[id=5495, 
name=OverseerHdfsCoreFailoverThread-96274232463130633-127.0.0.1:58236_solr-n_02,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at 
java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
 at java.lang.Thread.run(Thread.java:745)2) Thread[id=5493, 
name=OverseerStateUpdate-96274232463130633-127.0.0.1:58236_solr-n_02, 
state=TIMED_WAITING, group=Overseer state updater.] at 
sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164) 
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138)   
  at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:196)  
   at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE 
scope at org.apache.solr.cloud.TestRequestForwarding: 
   1) Thread[id=5495, 
name=OverseerHdfsCoreFailoverThread-96274232463130633-127.0.0.1:58236_solr-n_02,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=5493, 
name=OverseerStateUpdate-96274232463130633-127.0.0.1:58236_solr-n_02, 
state=TIMED_WAITING, group=Overseer state updater.]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164)
at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138)
at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:196)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([EBCB947803CDF279]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestRequestForwarding

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=5495, 
name=OverseerHdfsCoreFailoverThread-96274232463130633-127.0.0.1:58236_solr-n_02,
 state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at 
java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
 at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=5495, 
name=OverseerHdfsCoreFailoverThread-96274232463130633-127.0.0.1:58236_solr-n_02,
 state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([EBCB947803CDF279]:0)




Build Log:
[...truncated 11279 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestRequestForwarding
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_EBCB947803CDF279-001/init-core-data-001
   [junit4]   2> 652320 INFO  
(SUITE-TestRequestForwarding-seed#[EBCB947803CDF279]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 652322 INFO  
(TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[EBCB947803CDF279]) [ 
   ] o.a.s.SolrTestCaseJ4 ###Starting testMultiCollectionQuery
   [junit4]   2> 652323 INFO  
(TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[EBCB947803CDF279]) [ 
   ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 652323 INFO  (Thread-1359) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 652323 INFO  (Thread-1359) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 652423 INFO  

[jira] [Resolved] (LUCENE-7311) TermWeight shoud seek terms lazily

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-7311.
--
   Resolution: Fixed
Fix Version/s: 6.2
   master (7.0)

> TermWeight shoud seek terms lazily
> --
>
> Key: LUCENE-7311
> URL: https://issues.apache.org/jira/browse/LUCENE-7311
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Minor
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7311.patch, LUCENE-7311.patch
>
>
> Currently the terms are seeked eagerly in TermQuery.createWeight when 
> creating the TermContext. This might be wasteful when scores are not needed 
> since the query might be cached on some segments, thus seeking the term on 
> these segments is not needed. We could change TermWeight to only seek terms 
> in Weight.scorer when scores are not needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7311) TermWeight shoud seek terms lazily

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386069#comment-15386069
 ] 

ASF subversion and git services commented on LUCENE-7311:
-

Commit 71541bcd6cfd1e279faa1f2402403ac74cc5362d in lucene-solr's branch 
refs/heads/master from [~jpountz]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=71541bc ]

LUCENE-7311: Cached term queries do not seek the terms dictionary anymore.


> TermWeight shoud seek terms lazily
> --
>
> Key: LUCENE-7311
> URL: https://issues.apache.org/jira/browse/LUCENE-7311
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7311.patch, LUCENE-7311.patch
>
>
> Currently the terms are seeked eagerly in TermQuery.createWeight when 
> creating the TermContext. This might be wasteful when scores are not needed 
> since the query might be cached on some segments, thus seeking the term on 
> these segments is not needed. We could change TermWeight to only seek terms 
> in Weight.scorer when scores are not needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7311) TermWeight shoud seek terms lazily

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386068#comment-15386068
 ] 

ASF subversion and git services commented on LUCENE-7311:
-

Commit d5779335aa39b39e520849ccc01782880a3cfaaf in lucene-solr's branch 
refs/heads/branch_6x from [~jpountz]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d577933 ]

LUCENE-7311: Cached term queries do not seek the terms dictionary anymore.


> TermWeight shoud seek terms lazily
> --
>
> Key: LUCENE-7311
> URL: https://issues.apache.org/jira/browse/LUCENE-7311
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7311.patch, LUCENE-7311.patch
>
>
> Currently the terms are seeked eagerly in TermQuery.createWeight when 
> creating the TermContext. This might be wasteful when scores are not needed 
> since the query might be cached on some segments, thus seeking the term on 
> these segments is not needed. We could change TermWeight to only seek terms 
> in Weight.scorer when scores are not needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7036) Faster method for group.facet

[Danny Teichthal (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386056#comment-15386056
 ] 

Danny Teichthal commented on SOLR-7036:
---

[~jpswain] Hi, I'm a team member of [~mdvir1].
I suspect that the problem is that your field is defined with docValues=true.
Can you check again in your logs if you can find some error in the pattern 
"Type mismatc"  "was indexed as" ..., it should be indise 
class DocTermOrds.
If you also have the option to remove docValues=true from your field and test 
the new method again it would be grate.

In short, the new method does not support fields with doc values.
There's a related Lucene JIRA issue that describes that about UninvertedField - 
LUCENE-7119.

I see 2 options for a fix in the patch:
1. We should not allow using group.facet.method=uif on doc values field and 
throw a descriptive exception on this case.
2. In case we hit a doc value field - fallback to the original method.

Personally, I prefer to throw an exception, this way new consumers of this 
method will not get confused and will get a descriptive error.

The second option is used on the JSON facet API.
If you will facet on that field without grouping and set "facet.method=uif", 
then FacetField.createFacetProcessor will detect that it is a docValue and fall 
back to using the doc values processor instead of UIF.



Regarding the attached stacktrace:
I analyzed the stacktrace that you attached and it looks like there is only one 
active thread that is waiting on a monitor for the field value cache to be 
populated.
It is stuck on UnInvertedField.getUnInvertedField.
Since there's no other thread in work, it looks like there was another thread 
that tried to create the uninverted field and got an exception, in this case it 
will not notify other threads and thus you are stuck on wait().
I'm not familiar with the code that much, but it looks like it could be solved 
if the creation of UninvertedField was inside a try/finally block and 
cache.notifyAll(); was placed in the finally.

Would appreciate if you could confirm or deny this theory.



> Faster method for group.facet
> -
>
> Key: SOLR-7036
> URL: https://issues.apache.org/jira/browse/SOLR-7036
> Project: Solr
>  Issue Type: Improvement
>  Components: faceting
>Affects Versions: 4.10.3
>Reporter: Jim Musil
>Assignee: Erick Erickson
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-7036.patch, SOLR-7036.patch, SOLR-7036.patch, 
> SOLR-7036.patch, jstack-output.txt, performance.txt, source_for_patch.zip
>
>
> This is a patch that speeds up the performance of requests made with 
> group.facet=true. The original code that collects and counts unique facet 
> values for each group does not use the same improved field cache methods that 
> have been added for normal faceting in recent versions.
> Specifically, this approach leverages the UninvertedField class which 
> provides a much faster way to look up docs that contain a term. I've also 
> added a simple grouping map so that when a term is found for a doc, it can 
> quickly look up the group to which it belongs.
> Group faceting was very slow for our data set and when the number of docs or 
> terms was high, the latency spiked to multiple second requests. This solution 
> provides better overall performance -- from an average of 54ms to 32ms. It 
> also dropped our slowest performing queries way down -- from 6012ms to 991ms.
> I also added a few tests.
> I added an additional parameter so that you can choose to use this method or 
> the original. Add group.facet.method=fc to use the improved method or 
> group.facet.method=original which is the default if not specified.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6914) DecimalDigitFilter skips characters in some cases (supplemental?)

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386052#comment-15386052
 ] 

Adrien Grand commented on LUCENE-6914:
--

bq. now that we know the problem and can trivially reproduce with randomized 
tests, i'm not sure it's really relevant

I find explicit tests helpful because they are easier to dig into if they fail, 
so obvious problems can be more easily identified/fixed.

I think we should commit this patch, this filter is used in several of our 
language analyzers so this bug might be hitting quite a lot of users?

> DecimalDigitFilter skips characters in some cases (supplemental?)
> -
>
> Key: LUCENE-6914
> URL: https://issues.apache.org/jira/browse/LUCENE-6914
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.4
>Reporter: Hoss Man
> Attachments: LUCENE-6914.patch, LUCENE-6914.patch, LUCENE-6914.patch
>
>
> Found this while writing up the solr ref guide for DecimalDigitFilter. 
> With input like "ퟙퟡퟠퟜ" ("Double Struck" 1984) the filter produces "1ퟡ8ퟜ" (1, 
> double struck 9, 8, double struck 4)  add some non-decimal characters in 
> between the digits (ie: "ퟙxퟡxퟠxퟜ") and you get the expected output 
> ("1x9x8x4").  This doesn't affect all decimal characters though, as evident 
> by the existing test cases.
> Perhaps this is an off by one bug in the "if the original was supplementary, 
> shrink the string" code path?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9320) A REPLACENODE command to decommission an existing node with another new node

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386025#comment-15386025
 ] 

Erick Erickson commented on SOLR-9320:
--

(b) would copy the index twice, correct? I've seen (and I wouldn't lie) TB size 
indexes on a single replica so I think that would be prohibitively expensive.

> A REPLACENODE command to decommission an existing node with another new node
> 
>
> Key: SOLR-9320
> URL: https://issues.apache.org/jira/browse/SOLR-9320
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
>
> The command should accept a source node and target node. recreate the 
> replicas in source node in the target and do a DLETENODE of source node



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8589) Add aliases to the LIST action results in the Collections API

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated SOLR-8589:

Attachment: SOLR-8589.patch

Attaching new patch for master branch based on the existing one here and a 
little bit of Mark's patch on SOLR-4968.

Added flags to preserve existing behaviour and make printing aliases optional, 
added test.

> Add aliases to the LIST action results in the Collections API
> -
>
> Key: SOLR-8589
> URL: https://issues.apache.org/jira/browse/SOLR-8589
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-8589.patch, SOLR-8589.patch, 
> solr-8589-new-list-details-aliases.png
>
>
> Although it is possible to get a list of SolrCloud aliases vi an HTTP API, it 
> is not available as a typical query response, I believe it is only available 
> via the http API for zookeeper.
> The results from the LIST action in the Collections API is well-situated to 
> handle this. The current results are contained in a "collections" node, we 
> can simply add an "aliases" node if there are any aliases defined.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9106) Cache cluster properties in ZkStateReader

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386009#comment-15386009
 ] 

Alan Woodward commented on SOLR-9106:
-

bq. If we read the data just in time we always get the latest data

If you need this behaviour, you can use an explicit ClusterProperties object.  
ZkCLI and some tests already do this.  I'll add some javadocs to 
ZkStateReader.getClusterProperty() to make this more explicit though.

> Cache cluster properties in ZkStateReader
> -
>
> Key: SOLR-9106
> URL: https://issues.apache.org/jira/browse/SOLR-9106
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: master (7.0)
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: 6.1
>
> Attachments: SOLR-9106.patch, SOLR-9106.patch
>
>
> ZkStateReader currently makes calls into ZK every time getClusterProps() is 
> called.  Instead we should keep the data locally and use a Watcher to update 
> it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_92) - Build # 17311 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17311/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
IOException occured when talking to server at: 
http://127.0.0.1:44925/solr/testSolrCloudCollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: IOException 
occured when talking to server at: 
http://127.0.0.1:44925/solr/testSolrCloudCollection_shard1_replica1
at 
__randomizedtesting.SeedInfo.seed([39EE5F8B814E1A07:436F1A7B9A04477]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:739)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:193)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 

About SOLR-9310

[Pushkar Raste]

Hi,
https://issues.apache.org/jira/browse/SOLR-9310

PeerSync replication in SOLR seems to be completely broken since
fingerprint check was introduced. (or may be some other change made later,
I have not gone through git bisect process, to pin down commit that caused
the issue ).

I have documented my observation and have attached patch for fix and tests
to the ticket.


Can someone take a look at the issue. Let me know if I am missing something
here. Since this is a pretty big issue, I am surprised that no one has
noticed it yet and have not had much comments on the JIRA either.

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385977#comment-15385977
 ] 

Joel Bernstein commented on SOLR-9252:
--

One of the things that's really interesting about this ticket is that the 
algorithms used rely heavily on the statistics in the index. So, a search 
engine is really the best possible place to be doing machine learning on text 
documents. Logistic Regression is the easiest algorithm to get started with, 
but other algorithms will likely follow. 

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (64bit/jdk1.8.0_92) - Build # 332 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/332/
Java: 64bit/jdk1.8.0_92 -XX:-UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestRandomRequestDistribution.test

Error Message:
Shard a1x2_shard1_replica1 received all 10 requests

Stack Trace:
java.lang.AssertionError: Shard a1x2_shard1_replica1 received all 10 requests
at 
__randomizedtesting.SeedInfo.seed([306D13643B351859:B8392CBE95C975A1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.TestRandomRequestDistribution.testRequestTracking(TestRandomRequestDistribution.java:121)
at 
org.apache.solr.cloud.TestRandomRequestDistribution.test(TestRandomRequestDistribution.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 

[jira] [Comment Edited] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385955#comment-15385955
 ] 

Joel Bernstein edited comment on SOLR-9252 at 7/20/16 2:39 PM:
---

This is part of the larger ticket SOLR-9258, which will provide more context.

Here are some specifics about this ticket: 

Logistic regression is a machine learning classification algorithm.

It's binary, so it's used to determine if something belongs to a class or not.

With logistic regression you train a model using a training data set. And then 
use that model to classify other documents. 

This ticket trains a logistic regression model on text. So it builds a model 
based on the terms in the documents. New documents can then be classified based 
on the terms in the documents.

The terms in the document are known as *features*. 

The first step in the process is feature selection. Which is to select the 
important terms from the training set that will be used to build the model. 
This ticket uses an algorithm called Information Gain to select the features.

The next step is to train a model based on those features. This ticket uses 
Stochastic Gradient Descent to train a logistic regression model over the 
training set. Stochastic Gradient Descent is an iterative approach.

Both the features and the model can then be stored in a SolrCloud collection.













was (Author: joel.bernstein):
This is part of the larger ticket SOLR-9258, which will provide more context.

Here are some specifics about this ticket: 

Logistic regression is a machine learning classification algorithm.

It's binary, so it's used to determine if something belongs to a class or not.

With logistic regression you train a model using a training data set. And then 
use that model to classify other documents. 

This ticket trains a logistic regression model on text. So it builds a model 
based on the terms in the documents. New documents can then be classified based 
on the terms in the documents.

The terms in the document are known as *features*. 

The first step in the in process is feature selection. Which is to select the 
important terms from the training set that will be used to build the model. 
This ticket uses an algorithm called Information Gain to select the features.

The next step is to train a model based on those features. This ticket uses 
Stochastic Gradient Descent to train a logistic regression model over the 
training set. Stochastic Gradient Descent is an iterative approach.

Both the features and the model can then be stored in a SolrCloud collection.












> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will 

[jira] [Comment Edited] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385955#comment-15385955
 ] 

Joel Bernstein edited comment on SOLR-9252 at 7/20/16 2:37 PM:
---

This is part of the larger ticket SOLR-9258, which will provide more context.

Here are some specifics about this ticket: 

Logistic regression is a machine learning classification algorithm.

It's binary, so it's used to determine if something belongs to a class or not.

With logistic regression you train a model using a training data set. And then 
use that model to classify other documents. 

This ticket trains a logistic regression model on text. So it builds a model 
based on the terms in the documents. New documents can then be classified based 
on the terms in the documents.

The terms in the document are known as *features*. 

The first step in the in process is feature selection. Which is to select the 
important terms from the training set that will be used to build the model. 
This ticket uses an algorithm called Information Gain to select the features.

The next step is to train a model based on those features. This ticket uses 
Stochastic Gradient Descent to train a logistic regression model over the 
training set. Stochastic Gradient Descent is an iterative approach.

Both the features and the model can then be stored in a SolrCloud collection.













was (Author: joel.bernstein):
This is part of the larger ticket SOLR-9258, which will provide more context.

Here are some specifics about this ticket: 

Logistic regression is a machine learning classification algorithm.

It's binary, so it's used to determine if something belongs to a class or not.

With logistic regression you train a model using a training data set. And then 
use that model to classify other documents. 

This ticket trains a logistic regression model on text. So it builds a model 
based on the terms in the documents. New documents can then be classified based 
on the terms in the documents.

The terms in the document are known as *features*. 

The first step in the in process is feature selection. Which is to select the 
important terms from the training set that will be used to build the model. 
This ticket uses an algorithm called Information Gain to select the features.

The next step in is to train a model based on those features. This ticket uses 
Stochastic Gradient Descent to train a logistic regression model over the 
training set. Stochastic Gradient Descent is an iterative approach.

Both the features and the model can then be stored in a SolrCloud collection.












> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression 

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385955#comment-15385955
 ] 

Joel Bernstein commented on SOLR-9252:
--

This is part of the larger ticket SOLR-9258, which will provide more context.

Here are some specifics about this ticket: 

Logistic regression is a machine learning classification algorithm.

It's binary, so it's used to determine if something belongs to a class or not.

With logistic regression you train a model using a training data set. And then 
use that model to classify other documents. 

This ticket trains a logistic regression model on text. So it builds a model 
based on the terms in the documents. New documents can then be classified based 
on the terms in the documents.

The terms in the document are known as *features*. 

The first step in the in process is feature selection. Which is to select the 
important terms from the training set that will be used to build the model. 
This ticket uses an algorithm called Information Gain to select the features.

The next step in is to train a model based on those features. This ticket uses 
Stochastic Gradient Descent to train a logistic regression model over the 
training set. Stochastic Gradient Descent is an iterative approach.

Both the features and the model can then be stored in a SolrCloud collection.












> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5944) Support updates of numeric DocValues

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-5944:
---
Attachment: SOLR-5944.patch

bq. it is acquired by indexing threads who only want to read stuff from update 
log
Ah, I see. That makes it clear for me; I've updated the patch now and have done 
the wait loop as per your suggestion.

bq. In your latest patch, acquiring the read lock to call versionAdd is not 
necessary – it will do that anyway. You can re-acquire it for reading the 
version after the method call returns.
Done, removed.

bq. I don't think the case of vinfo.lookupVersion returning a negative value 
(for deletes) is handled here at all.
Indeed, I hadn't handled it! I've now handled it in this new patch.

bq. What happens if the document has been deleted already (due to reordering on 
the replica) when you enter waitForDependentUpdates? i.e. what if re-ordering 
leads to new_doc (v1) -> del_doc (v3) -> dv_update (v2) on the replica?
I think it should now be fixed: v2 update would be dropped since abs(v3) > v2 
(earlier I was just checking v3 > v2, which didn't consider negative versions). 
I'll add a test for this.

bq. Similarly, how do we handle the case when the doc has been deleted on the 
leader when you execute fetchMissingUpdateFromLeader. Does RTG return the 
requested version even if the doc has been deleted already? I suspect it does 
but be nice to confirm.
I think as long as a particular update for the given version is present at the 
leader's ulog/tlog, it will be returned. I will add a test for this scenario 
specifically anyway.

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9275) make XML Query Parser support extensible-via-configuration

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385941#comment-15385941
 ] 

ASF subversion and git services commented on SOLR-9275:
---

Commit 03eb44ad121777f38d97f40862ab03b71207a5df in lucene-solr's branch 
refs/heads/branch_5x from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=03eb44a ]

SOLR-9275: make XML QueryParser support (defType=xmlparser) extensible via 
configuration


> make XML Query Parser support extensible-via-configuration
> --
>
> Key: SOLR-9275
> URL: https://issues.apache.org/jira/browse/SOLR-9275
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9275.patch
>
>
> SOLR-839 added XML QueryParser support (deftype=xmlparser) and this ticket 
> here proposes to make that support extensible-via-configuration.
> Objective:
>  * To support use of custom query builders.
>  * To support use of custom query builders _without_ a corresponding custom 
> XmlQParser plugin class.
> Illustration:
>  * solrconfig.xml snippet to configure use of the custom builders
> {code}
> 
>   org.apache.solr.search.HelloQueryBuilder
>   org.apache.solr.search.GoodbyeQueryBuilder
> 
> {code}
> * HelloQueryBuilder and GoodbyeQueryBuilder both extend the new abstract 
> SolrQueryBuilder class.
> {code}
> + public abstract class SolrQueryBuilder implements QueryBuilder {
> +   protected final SolrQueryRequest req;
> +   protected final QueryBuilder queryFactory;
> +   public SolrQueryBuilder(String defaultField, Analyzer analyzer,
> +   SolrQueryRequest req, QueryBuilder queryFactory) {
> + this.req = req;
> + this.queryFactory = queryFactory;
> +   }
> + }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9275) make XML Query Parser support extensible-via-configuration

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385940#comment-15385940
 ] 

ASF subversion and git services commented on SOLR-9275:
---

Commit d9dcd5a0758b3526e08393b77b8a48adfedbf5a4 in lucene-solr's branch 
refs/heads/branch_5x from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d9dcd5a ]

SOLR-9275: add defaultField protected field to queryparser/xml's CoreParser


> make XML Query Parser support extensible-via-configuration
> --
>
> Key: SOLR-9275
> URL: https://issues.apache.org/jira/browse/SOLR-9275
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9275.patch
>
>
> SOLR-839 added XML QueryParser support (deftype=xmlparser) and this ticket 
> here proposes to make that support extensible-via-configuration.
> Objective:
>  * To support use of custom query builders.
>  * To support use of custom query builders _without_ a corresponding custom 
> XmlQParser plugin class.
> Illustration:
>  * solrconfig.xml snippet to configure use of the custom builders
> {code}
> 
>   org.apache.solr.search.HelloQueryBuilder
>   org.apache.solr.search.GoodbyeQueryBuilder
> 
> {code}
> * HelloQueryBuilder and GoodbyeQueryBuilder both extend the new abstract 
> SolrQueryBuilder class.
> {code}
> + public abstract class SolrQueryBuilder implements QueryBuilder {
> +   protected final SolrQueryRequest req;
> +   protected final QueryBuilder queryFactory;
> +   public SolrQueryBuilder(String defaultField, Analyzer analyzer,
> +   SolrQueryRequest req, QueryBuilder queryFactory) {
> + this.req = req;
> + this.queryFactory = queryFactory;
> +   }
> + }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9275) make XML Query Parser support extensible-via-configuration

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385942#comment-15385942
 ] 

ASF subversion and git services commented on SOLR-9275:
---

Commit bb2f70eabf3fcb63662b3580936dbce6c7df1449 in lucene-solr's branch 
refs/heads/branch_5x from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bb2f70e ]

SOLR-9275: fix NPE in SolrCoreParser.init


> make XML Query Parser support extensible-via-configuration
> --
>
> Key: SOLR-9275
> URL: https://issues.apache.org/jira/browse/SOLR-9275
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9275.patch
>
>
> SOLR-839 added XML QueryParser support (deftype=xmlparser) and this ticket 
> here proposes to make that support extensible-via-configuration.
> Objective:
>  * To support use of custom query builders.
>  * To support use of custom query builders _without_ a corresponding custom 
> XmlQParser plugin class.
> Illustration:
>  * solrconfig.xml snippet to configure use of the custom builders
> {code}
> 
>   org.apache.solr.search.HelloQueryBuilder
>   org.apache.solr.search.GoodbyeQueryBuilder
> 
> {code}
> * HelloQueryBuilder and GoodbyeQueryBuilder both extend the new abstract 
> SolrQueryBuilder class.
> {code}
> + public abstract class SolrQueryBuilder implements QueryBuilder {
> +   protected final SolrQueryRequest req;
> +   protected final QueryBuilder queryFactory;
> +   public SolrQueryBuilder(String defaultField, Analyzer analyzer,
> +   SolrQueryRequest req, QueryBuilder queryFactory) {
> + this.req = req;
> + this.queryFactory = queryFactory;
> +   }
> + }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385902#comment-15385902
 ] 

David Smiley commented on SOLR-9252:


Can the point of this be explained in laymans terms?  I am not familiar with 
logistic regression and how it relates to search.

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9323) Add a new method getCollectionNamesFast() to ClusterSate which returns unverified list of collection names

[Noble Paul (JIRA)]

Noble Paul created SOLR-9323:


 Summary: Add a new method getCollectionNamesFast() to ClusterSate 
which returns unverified list of collection names
 Key: SOLR-9323
 URL: https://issues.apache.org/jira/browse/SOLR-9323
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul
Assignee: Noble Paul


Currently the {{getCollections()}} method is deprecated and everyone is forced 
to use {{getCollectionsMap()}} which could be extremely expensive depending on 
the no:of collections. The {{getCollectionsMap()}} method is very expensive and 
should never be invoked on each request



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9252) Feature selection and logistic regression on text

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-9252:
---
Attachment: SOLR-9252.patch

It turn out that, the cause of test fail is we create temporary collection and 
do not delete it. So the exception is thrown when we wanna create same 
temporary collection in another test.

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_92) - Build # 5995 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/5995/
Java: 64bit/jdk1.8.0_92 -XX:+UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrVersionReplicationTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [InternalHttpClient]
at __randomizedtesting.SeedInfo.seed([A76FFEFA12D2F20A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:62251/forceleader_test_collection_shard1_replica3]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:62251/forceleader_test_collection_shard1_replica3]
at 
__randomizedtesting.SeedInfo.seed([A76FFEFA12D2F20A:41F8CA3A2B500B6B]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:739)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 

[jira] [Commented] (SOLR-7495) Unexpected docvalues type NUMERIC when grouping by a int facet

[Scott Stults (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385813#comment-15385813
 ] 

Scott Stults commented on SOLR-7495:


The patch uploaded in April works for me on the 6.1 branch. Is it not working 
for you?

> Unexpected docvalues type NUMERIC when grouping by a int facet
> --
>
> Key: SOLR-7495
> URL: https://issues.apache.org/jira/browse/SOLR-7495
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.3
>Reporter: Fabio Batista da Silva
> Attachments: SOLR-7495.patch, SOLR-7495.patch
>
>
> Hey All,
> After upgrading from solr 4.10 to 5.1 with solr could
> I'm getting a IllegalStateException when i try to facet a int field.
> IllegalStateException: unexpected docvalues type NUMERIC for field 'year' 
> (expected=SORTED). Use UninvertingReader or index with docvalues.
> schema.xml
> {code}
> 
> 
> 
> 
> 
> 
>  multiValued="false" required="true"/>
>  multiValued="false" required="true"/>
> 
> 
>  stored="true"/>
> 
> 
> 
>  />
>  sortMissingLast="true"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  precisionStep="0" positionIncrementGap="0"/>
>  positionIncrementGap="0"/>
>  positionIncrementGap="100">
> 
> 
>  words="stopwords.txt" />
> 
>  maxGramSize="15"/>
> 
> 
> 
>  words="stopwords.txt" />
>  synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
> 
> 
> 
>  positionIncrementGap="100">
> 
> 
>  words="stopwords.txt" />
> 
>  maxGramSize="15"/>
> 
> 
> 
>  words="stopwords.txt" />
>  synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
> 
> 
> 
>  class="solr.SpatialRecursivePrefixTreeFieldType" geo="true" 
> distErrPct="0.025" maxDistErr="0.09" units="degrees" />
> 
> id
> name
> 
> 
> {code}
> query :
> {code}
> http://solr.dev:8983/solr/my_collection/select?wt=json=id=index_type:foobar=true=year_make_model=true=true=year
> {code}
> Exception :
> {code}
> ull:org.apache.solr.common.SolrException: Exception during facet.field: year
> at org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:627)
> at org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:612)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:566)
> at 
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:637)
> at 
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:280)
> at 
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:106)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:222)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at 
> 

[JENKINS] Lucene-Solr-Tests-master - Build # 1281 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1281/

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.overseer.ZkStateWriterTest

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.overseer.ZkStateWriterTest: 1) Thread[id=7881, 
name=watches-1239-thread-1, state=TIMED_WAITING, group=TGRP-ZkStateWriterTest]  
   at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.overseer.ZkStateWriterTest: 
   1) Thread[id=7881, name=watches-1239-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([4F73AC884D3AD444]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.overseer.ZkStateWriterTest

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=7881, name=watches-1239-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest] at sun.misc.Unsafe.park(Native Method)
 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)  
   at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=7881, name=watches-1239-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([4F73AC884D3AD444]:0)




Build Log:
[...truncated 11445 lines...]
   [junit4] Suite: org.apache.solr.cloud.overseer.ZkStateWriterTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.overseer.ZkStateWriterTest_4F73AC884D3AD444-001/init-core-data-001
   [junit4]   2> 980584 INFO  
(SUITE-ZkStateWriterTest-seed#[4F73AC884D3AD444]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 980586 INFO  
(TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[4F73AC884D3AD444])
 [] o.a.s.SolrTestCaseJ4 ###Starting testExternalModificationToStateFormat2

[jira] [Commented] (SOLR-9197) Audit all solrconfig components and ensure that they work with the Config APIs

[Alex D (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385779#comment-15385779
 ] 

Alex D commented on SOLR-9197:
--

My issue may be related to this Jira.  

I tried adding a highlight search component using the Config API and it is not 
picked up by Solr.  I can see my new highlight search component when using 
Solr's "/solr//config" however the request handler is using the 
default highlighting values instead of the ones I specified.  

I can post the relevant configuration if it helps.

> Audit all solrconfig components and ensure that they work with the Config APIs
> --
>
> Key: SOLR-9197
> URL: https://issues.apache.org/jira/browse/SOLR-9197
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>
> Components assume that the configuration is read from {{solrconfig.xml}} and 
> all values are NamedList. Config APIs  use and store JSON and the syntax is 
> different. Components should work with both syntax.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9275) make XML Query Parser support extensible-via-configuration

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385728#comment-15385728
 ] 

ASF subversion and git services commented on SOLR-9275:
---

Commit a4970c8685899d1262320b056992874dd672a40f in lucene-solr's branch 
refs/heads/branch_6x from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a4970c8 ]

SOLR-9275: fix NPE in SolrCoreParser.init


> make XML Query Parser support extensible-via-configuration
> --
>
> Key: SOLR-9275
> URL: https://issues.apache.org/jira/browse/SOLR-9275
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9275.patch
>
>
> SOLR-839 added XML QueryParser support (deftype=xmlparser) and this ticket 
> here proposes to make that support extensible-via-configuration.
> Objective:
>  * To support use of custom query builders.
>  * To support use of custom query builders _without_ a corresponding custom 
> XmlQParser plugin class.
> Illustration:
>  * solrconfig.xml snippet to configure use of the custom builders
> {code}
> 
>   org.apache.solr.search.HelloQueryBuilder
>   org.apache.solr.search.GoodbyeQueryBuilder
> 
> {code}
> * HelloQueryBuilder and GoodbyeQueryBuilder both extend the new abstract 
> SolrQueryBuilder class.
> {code}
> + public abstract class SolrQueryBuilder implements QueryBuilder {
> +   protected final SolrQueryRequest req;
> +   protected final QueryBuilder queryFactory;
> +   public SolrQueryBuilder(String defaultField, Analyzer analyzer,
> +   SolrQueryRequest req, QueryBuilder queryFactory) {
> + this.req = req;
> + this.queryFactory = queryFactory;
> +   }
> + }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9275) make XML Query Parser support extensible-via-configuration

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385727#comment-15385727
 ] 

ASF subversion and git services commented on SOLR-9275:
---

Commit 80a2e73ccd8498b4714347268987a042867cd65f in lucene-solr's branch 
refs/heads/branch_6x from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=80a2e73 ]

SOLR-9275: make XML QueryParser support (defType=xmlparser) extensible via 
configuration


> make XML Query Parser support extensible-via-configuration
> --
>
> Key: SOLR-9275
> URL: https://issues.apache.org/jira/browse/SOLR-9275
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9275.patch
>
>
> SOLR-839 added XML QueryParser support (deftype=xmlparser) and this ticket 
> here proposes to make that support extensible-via-configuration.
> Objective:
>  * To support use of custom query builders.
>  * To support use of custom query builders _without_ a corresponding custom 
> XmlQParser plugin class.
> Illustration:
>  * solrconfig.xml snippet to configure use of the custom builders
> {code}
> 
>   org.apache.solr.search.HelloQueryBuilder
>   org.apache.solr.search.GoodbyeQueryBuilder
> 
> {code}
> * HelloQueryBuilder and GoodbyeQueryBuilder both extend the new abstract 
> SolrQueryBuilder class.
> {code}
> + public abstract class SolrQueryBuilder implements QueryBuilder {
> +   protected final SolrQueryRequest req;
> +   protected final QueryBuilder queryFactory;
> +   public SolrQueryBuilder(String defaultField, Analyzer analyzer,
> +   SolrQueryRequest req, QueryBuilder queryFactory) {
> + this.req = req;
> + this.queryFactory = queryFactory;
> +   }
> + }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 278 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/278/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates: 1) 
Thread[id=11522, 
name=OverseerHdfsCoreFailoverThread-96273155570139145-127.0.0.1:62964_solr-n_02,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at 
java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
 at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates: 
   1) Thread[id=11522, 
name=OverseerHdfsCoreFailoverThread-96273155570139145-127.0.0.1:62964_solr-n_02,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([A6E11BE7B4E628ED]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=11522, 
name=OverseerHdfsCoreFailoverThread-96273155570139145-127.0.0.1:62964_solr-n_02,
 state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at 
java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
 at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=11522, 
name=OverseerHdfsCoreFailoverThread-96273155570139145-127.0.0.1:62964_solr-n_02,
 state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([A6E11BE7B4E628ED]:0)




Build Log:
[...truncated 11444 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates
   [junit4]   2> Creating dataDir: 
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_A6E11BE7B4E628ED-001/init-core-data-001
   [junit4]   2> 1475429 INFO  
(SUITE-TestStressCloudBlindAtomicUpdates-seed#[A6E11BE7B4E628ED]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=SSL overhead seems to cause 
OutOfMemory when stress testing)
   [junit4]   2> 1475430 INFO  
(SUITE-TestStressCloudBlindAtomicUpdates-seed#[A6E11BE7B4E628ED]-worker) [] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1475430 INFO  (Thread-3275) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1475430 INFO  (Thread-3275) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1475530 INFO  
(SUITE-TestStressCloudBlindAtomicUpdates-seed#[A6E11BE7B4E628ED]-worker) [] 
o.a.s.c.ZkTestServer start zk server on port:53270
   [junit4]   2> 1475531 INFO  
(SUITE-TestStressCloudBlindAtomicUpdates-seed#[A6E11BE7B4E628ED]-worker) [] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1475531 INFO  
(SUITE-TestStressCloudBlindAtomicUpdates-seed#[A6E11BE7B4E628ED]-worker) [] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1475534 INFO  (zkCallback-1754-thread-1) [] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@124ff7f9 
name:ZooKeeperConnection Watcher:127.0.0.1:53270 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1475534 INFO  
(SUITE-TestStressCloudBlindAtomicUpdates-seed#[A6E11BE7B4E628ED]-worker) [] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1475534 INFO  
(SUITE-TestStressCloudBlindAtomicUpdates-seed#[A6E11BE7B4E628ED]-worker) [] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1475534 INFO  
(SUITE-TestStressCloudBlindAtomicUpdates-seed#[A6E11BE7B4E628ED]-worker) [] 
o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 1475541 INFO  (jetty-launcher-1753-thread-1) [] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1475543 INFO  (jetty-launcher-1753-thread-4) [] 

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 729 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/729/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatchesWorkForStateFormat1

Error Message:
CollectionStateWatcher not notified of stateformat=1 collection creation

Stack Trace:
java.lang.AssertionError: CollectionStateWatcher not notified of stateformat=1 
collection creation
at 
__randomizedtesting.SeedInfo.seed([B955BFFFB05B2E84:DE92A31144E30A25]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatchesWorkForStateFormat1(TestCollectionStateWatchers.java:267)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13028 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating 

[jira] [Commented] (SOLR-9322) A new RESHARD command to increase no:of shards to have uniform sized shards

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385583#comment-15385583
 ] 

Noble Paul commented on SOLR-9322:
--

The command would have the following parameters

 * collection :  (required) 
 * increment : No:of shards to be added. Either increment of decrement must be 
provided
 * decrement: No:of shards to be removed

> A new RESHARD command to increase no:of shards to have uniform sized shards
> ---
>
> Key: SOLR-9322
> URL: https://issues.apache.org/jira/browse/SOLR-9322
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>
> The SPLITSHARD is a rather blunt tool. It looks fine when you have very few 
> no:of shards. If we have a very large no:of shards and we need to scale out 
> by increasing the shard count by a smaller percentage (say 20% increase) it 
> becomes rather useless.To achieve this we need to split the existing shards 
> more smartly in such a way that the resulting shard set will have roughly 
> similar no:of docs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9241) Rebalance API for SolrCloud

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385578#comment-15385578
 ] 

Noble Paul commented on SOLR-9241:
--

The autosharding feature needs a more comprehensive change. I have opened 
SOLR93-22 to capture that

> Rebalance API for SolrCloud
> ---
>
> Key: SOLR-9241
> URL: https://issues.apache.org/jira/browse/SOLR-9241
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 6.1
> Environment: Ubuntu, Mac OsX
>Reporter: Nitin Sharma
>  Labels: Cluster, SolrCloud
> Fix For: 6.1
>
> Attachments: Redistribute_After.jpeg, Redistribute_Before.jpeg, 
> Redistribute_call.jpeg, Replace_After.jpeg, Replace_Before.jpeg, 
> Replace_Call.jpeg, SOLR-9241-4.6.patch, SOLR-9241-6.1.patch
>
>   Original Estimate: 2,016h
>  Remaining Estimate: 2,016h
>
> This is the v1 of the patch for Solrcloud Rebalance api (as described in 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/) , built at 
> Bloomreach by Nitin Sharma and Suruchi Shah. The goal of the API  is to 
> provide a zero downtime mechanism to perform data manipulation and  efficient 
> core allocation in solrcloud. This API was envisioned to be the base layer 
> that enables Solrcloud to be an auto scaling platform. (and work in unison 
> with other complementing monitoring and scaling features).
> Patch Status:
> ===
> The patch is work in progress and incremental. We have done a few rounds of 
> code clean up. We wanted to get the patch going first to get initial feed 
> back.  We will continue to work on making it more open source friendly and 
> easily testable.
>  Deployment Status:
> 
> The platform is deployed in production at bloomreach and has been battle 
> tested for large scale load. (millions of documents and hundreds of 
> collections).
>  Internals:
> =
> The internals of the API and performance : 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/
> It is built on top of the admin collections API as an action (with various 
> flavors). At a high level, the rebalance api provides 2 constructs:
> Scaling Strategy:  Decides how to move the data.  Every flavor has multiple 
> options which can be reviewed in the api spec.
> Re-distribute  - Move around data in the cluster based on capacity/allocation.
> Auto Shard  - Dynamically shard a collection to any size.
> Smart Merge - Distributed Mode - Helps merging data from a larger shard setup 
> into smaller one.  (the source should be divisible by destination)
> Scale up -  Add replicas on the fly
> Scale Down - Remove replicas on the fly
> Allocation Strategy:  Decides where to put the data.  (Nodes with least 
> cores, Nodes that do not have this collection etc). Custom implementations 
> can be built on top as well. One other example is Availability Zone aware. 
> Distribute data such that every replica is placed on different availability 
> zone to support HA.
>  Detailed API Spec:
> 
>   https://github.com/bloomreach/solrcloud-rebalance-api
>  Contributors:
> =
>   Nitin Sharma
>   Suruchi Shah
>  Questions/Comments:
> =
>   You can reach me at nitin.sha...@bloomreach.com



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9322) A new RESHARD command to increase no:of shards to have uniform sized shards

[Noble Paul (JIRA)]

Noble Paul created SOLR-9322:


 Summary: A new RESHARD command to increase no:of shards to have 
uniform sized shards
 Key: SOLR-9322
 URL: https://issues.apache.org/jira/browse/SOLR-9322
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul


The SPLITSHARD is a rather blunt tool. It looks fine when you have very few 
no:of shards. If we have a very large no:of shards and we need to scale out by 
increasing the shard count by a smaller percentage (say 20% increase) it 
becomes rather useless.To achieve this we need to split the existing shards 
more smartly in such a way that the resulting shard set will have roughly 
similar no:of docs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_92) - Build # 17308 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17308/
Java: 64bit/jdk1.8.0_92 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test

Error Message:
Exactly one shard should have changed, instead: [core_node1, core_node2] 
nodes=([expiry_shard2_replica1(core_node2), 
expiry_shard1_replica1(core_node1)]) expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: Exactly one shard should have changed, instead: 
[core_node1, core_node2] nodes=([expiry_shard2_replica1(core_node2), 
expiry_shard1_replica1(core_node1)]) expected:<1> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([604F62D15C91A840:E81B5D0BF26DC5B8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test(DistribDocExpirationUpdateProcessorTest.java:116)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   

[jira] [Comment Edited] (SOLR-9319) DELETEREPLICA should be able to accept just count and remove replicas intelligenty

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384324#comment-15384324
 ] 

Noble Paul edited comment on SOLR-9319 at 7/20/16 7:19 AM:
---

The old functionality continues to be there. This kicks in when 'count' is 
specified as a parameter


was (Author: noble.paul):
The old functionality continues to be there. This kicks in when 'count' is 
specified a parameter

> DELETEREPLICA should be able to accept  just count and remove replicas 
> intelligenty
> ---
>
> Key: SOLR-9319
> URL: https://issues.apache.org/jira/browse/SOLR-9319
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3422 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3422/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testCanWaitForNonexistantCollection

Error Message:
waitForState was not triggered by collection creation

Stack Trace:
java.lang.AssertionError: waitForState was not triggered by collection creation
at 
__randomizedtesting.SeedInfo.seed([E4122F559A4CA2A6:4F32B4E3F712668D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testCanWaitForNonexistantCollection(TestCollectionStateWatchers.java:182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13013 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating dataDir: 

[jira] [Commented] (LUCENE-7381) Add new RangeField

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385420#comment-15385420
 ] 

Adrien Grand commented on LUCENE-7381:
--

The bounds checks for getMin and getMax apply to {{type.pointDimensionCount()}} 
but it really should be {{type.pointDimensionCount()/2}}?
I think I would find it a bit easier to read if each query type had a different 
RangeFieldComparator impl, the fact that all cases are handled in a single 
method makes it a bit hard to follow for me.

> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch, 
> LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing 
> numeric ranges that could be useful for a number of applications.
> For example, a single dimension represents a span along a single axis such as 
> indexing calendar entries start and end time, 2d range could represent 
> bounding boxes for geometric applications (e.g., supporting Point based geo 
> shapes), 3d ranges bounding cubes for 3d geometric applications (collision 
> detection, 3d geospatial), and 4d ranges for space time applications. I'm 
> sure there's applicability for 5d+ ranges but a first incarnation should 
> likely limit for performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7386) Flatten nested disjunctions

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385415#comment-15385415
 ] 

David Smiley commented on LUCENE-7386:
--

Yes, thanks; I've yet to use that.

> Flatten nested disjunctions
> ---
>
> Key: LUCENE-7386
> URL: https://issues.apache.org/jira/browse/LUCENE-7386
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7386.patch, LUCENE-7386.patch
>
>
> Now that coords are gone it became easier to flatten nested disjunctions. It 
> might sound weird to write nested disjunctions in the first place, but 
> disjunctions can be created implicitly by other queries such as 
> more-like-this, LatLonPoint.newBoxQuery, non-scoring synonym queries, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org