date:20190124

[jira] [Updated] (SOLR-13139) RefGuide: fix date range syntax

2019-01-24 Thread Mikhail Khludnev (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-13139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-13139:

Fix Version/s: 7.7

> RefGuide: fix date range syntax 
> 
>
> Key: SOLR-13139
> URL: https://issues.apache.org/jira/browse/SOLR-13139
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Fix For: 8.0, 7.7, master (9.0)
>
> Attachments: SOLR-13139.patch
>
>
> https://lucene.apache.org/solr/guide/7_6/working-with-dates.html suggests 
> {{2000-11T13}} as a date range that makes to sense. It's worth to suggest the 
> correct syntax at least, not sure about adding explanation or validating 
> delimiters. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-13029) Allow HDFS backup/restore buffer size to be configured

2019-01-24 Thread Mikhail Khludnev (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev reassigned SOLR-13029:
---

Assignee: Mikhail Khludnev

> Allow HDFS backup/restore buffer size to be configured
> --
>
> Key: SOLR-13029
> URL: https://issues.apache.org/jira/browse/SOLR-13029
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, hdfs
>Affects Versions: 7.5, 8.0
>Reporter: Tim Owen
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-13029.patch, SOLR-13029.patch, SOLR-13029.patch
>
>
> There's a default hardcoded buffer size setting of 4096 in the HDFS code 
> which means in particular that restoring a backup from HDFS takes a long 
> time. Copying multi-GB files from HDFS using a buffer as small as 4096 bytes 
> is very inefficient. We changed this in our local build used in production to 
> 256kB and saw a 10x speed improvement when restoring a backup. Attached patch 
> simply makes this size configurable using a command line setting, much like 
> several other buffer size values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13156) Limiting field facet with certain terms via {!terms} not taking into account sorting

2019-01-24 Thread Lucene/Solr QA (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751881#comment-16751881
 ] 

Lucene/Solr QA commented on SOLR-13156:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  1m 41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  1m 41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  1m 41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 39m 
59s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 55s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13156 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12956140/SOLR-13156.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon 
Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / c317119 |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | 1.8.0_191 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/268/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/268/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Limiting field facet with certain terms via {!terms} not taking into account 
> sorting
> 
>
> Key: SOLR-13156
> URL: https://issues.apache.org/jira/browse/SOLR-13156
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Konstantin Perikov
>Priority: Major
> Attachments: SOLR-13156.patch
>
>
> When I'm doing limiting facet keys with \{!terms} it doesn't take into 
> account sorting.
> First query not limiting the facet keys:
> {{facet.field=title=count=on=*:*}}
> Response as expected:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "book2",3, "book1",2, "book3",1]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
>  
> When doing it with limiting:
> {{facet.field=\{!terms=Book3,Book2,Book1}title=count=on=*:*}}
> I'm getting the exact order of how I list terms:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "Book3",1, "Book2",3, "Book1",2]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
> I've looked at the code, and it's clearly an issue there:
>  
> org.apache.solr.request.SimpleFacets#getListedTermCounts
>  
> {{for (String term : terms) {}}
> {{    int count = searcher.numDocs(ft.getFieldQuery(null, sf, term), 
> parsed.docs);}}
> {{    res.add(term, count);}}
> {{}}}
>  
> it's just basically iterating over terms and don't do any sorting at all. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-13130) during the ResponseBuilder.STAGE_GET_FIELDS avoid creation of Strings

2019-01-24 Thread Noble Paul (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-13130:
-

Assignee: Noble Paul

> during the ResponseBuilder.STAGE_GET_FIELDS avoid creation of Strings
> -
>
> Key: SOLR-13130
> URL: https://issues.apache.org/jira/browse/SOLR-13130
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> The javabin format bytes can be directly written out to the client instead of 
> doing the double transformation
>  
>  
> {{ utf-8 -> String -> utf8 }}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12885) BinaryResponseWriter (javabin format) should directly copy from Bytesref to output

2019-01-24 Thread Noble Paul (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-12885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul resolved SOLR-12885.
---
   Resolution: Fixed
Fix Version/s: 8.0

> BinaryResponseWriter (javabin format) should directly copy from Bytesref to 
> output
> --
>
> Key: SOLR-12885
> URL: https://issues.apache.org/jira/browse/SOLR-12885
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: 8.0
>
> Attachments: SOLR-12885.patch, SOLR-12885.patch, SOLR-12885.patch, 
> SOLR-12885.patch, SOLR-12885.patch
>
>
> The format format in which bytes are stored in {{BytesRef}} and the javabin 
> string format are both the same. We don't need to convert the string/text 
> fields from {{BytesRef}} to String and back to UTF8 
> {{Now a String/Text field is read and written out as follows}}
> {{luceneindex(UTF8 bytes) --> UTF16 (char[]) --> new String() a copy of UTF16 
> char[] -->  UTF8bytes(javabin format)}}
> This does not add a new type to javabin. It's encoded as String in the 
> serialized data. When it is deserialized, you get a String back



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12924) Optimize JavabinCodec to use byte[] backed CharSequence instead of String

2019-01-24 Thread Noble Paul (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul resolved SOLR-12924.
---
   Resolution: Fixed
Fix Version/s: 8.0

> Optimize JavabinCodec to use byte[] backed CharSequence instead of String
> -
>
> Key: SOLR-12924
> URL: https://issues.apache.org/jira/browse/SOLR-12924
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Priority: Major
> Fix For: 8.0
>
>
> javabin format uses {{UTF8}} format for String serialization and java Strings 
> are {{UTF16}} and java String always days a buffer copy of those {{UTF16}} 
> input.
> So, what I propose is 
> * we create a byte[] backed {{CharSequence}} implementation say 
> {{Utf8CharSequence}}.
> * Optionally, javabinCodec can should be able to return this 
> {{Utf8CharSequence}} instead of all Strings
> * If a {{charAt()}} or {{toString()}} call is made to {{Utf8CharSequence}} it 
> creates a {{String}} and cache it
> * When this {{Utf8CharSequence}} needs to be serialized again in {{javabin}} 
> format it should copy the underlying utf8 byte[]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-12924) Optimize JavabinCodec to use byte[] backed CharSequence instead of String

2019-01-24 Thread Noble Paul (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-12924:
-

Assignee: Noble Paul

> Optimize JavabinCodec to use byte[] backed CharSequence instead of String
> -
>
> Key: SOLR-12924
> URL: https://issues.apache.org/jira/browse/SOLR-12924
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: 8.0
>
>
> javabin format uses {{UTF8}} format for String serialization and java Strings 
> are {{UTF16}} and java String always days a buffer copy of those {{UTF16}} 
> input.
> So, what I propose is 
> * we create a byte[] backed {{CharSequence}} implementation say 
> {{Utf8CharSequence}}.
> * Optionally, javabinCodec can should be able to return this 
> {{Utf8CharSequence}} instead of all Strings
> * If a {{charAt()}} or {{toString()}} call is made to {{Utf8CharSequence}} it 
> creates a {{String}} and cache it
> * When this {{Utf8CharSequence}} needs to be serialized again in {{javabin}} 
> format it should copy the underlying utf8 byte[]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-8.x-Linux (64bit/jdk-12-ea+23) - Build # 79 - Unstable!

2019-01-24 Thread Policeman Jenkins Server

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/79/
Java: 64bit/jdk-12-ea+23 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestCloudRecovery2.test

Error Message:
 Timeout waiting to see state for collection=collection1 
:DocCollection(collection1//collections/collection1/state.json/15)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node3":{   "core":"collection1_shard1_replica_n1",   
"base_url":"http://127.0.0.1:40873/solr;,   
"node_name":"127.0.0.1:40873_solr",   "state":"active",   
"type":"NRT",   "force_set_state":"false",   "leader":"true"},  
   "core_node4":{   "core":"collection1_shard1_replica_n2", 
  "base_url":"http://127.0.0.1:33785/solr;,   
"node_name":"127.0.0.1:33785_solr",   "state":"down",   
"type":"NRT",   "force_set_state":"false",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"2",   
"autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"} Live 
Nodes: [127.0.0.1:33785_solr, 127.0.0.1:40873_solr] Last available state: 
DocCollection(collection1//collections/collection1/state.json/15)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node3":{   "core":"collection1_shard1_replica_n1",   
"base_url":"http://127.0.0.1:40873/solr;,   
"node_name":"127.0.0.1:40873_solr",   "state":"active",   
"type":"NRT",   "force_set_state":"false",   "leader":"true"},  
   "core_node4":{   "core":"collection1_shard1_replica_n2", 
  "base_url":"http://127.0.0.1:33785/solr;,   
"node_name":"127.0.0.1:33785_solr",   "state":"down",   
"type":"NRT",   "force_set_state":"false",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"2",   
"autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: 
Timeout waiting to see state for collection=collection1 
:DocCollection(collection1//collections/collection1/state.json/15)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node3":{
  "core":"collection1_shard1_replica_n1",
  "base_url":"http://127.0.0.1:40873/solr;,
  "node_name":"127.0.0.1:40873_solr",
  "state":"active",
  "type":"NRT",
  "force_set_state":"false",
  "leader":"true"},
"core_node4":{
  "core":"collection1_shard1_replica_n2",
  "base_url":"http://127.0.0.1:33785/solr;,
  "node_name":"127.0.0.1:33785_solr",
  "state":"down",
  "type":"NRT",
  "force_set_state":"false",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
Live Nodes: [127.0.0.1:33785_solr, 127.0.0.1:40873_solr]
Last available state: 
DocCollection(collection1//collections/collection1/state.json/15)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node3":{
  "core":"collection1_shard1_replica_n1",
  "base_url":"http://127.0.0.1:40873/solr;,
  "node_name":"127.0.0.1:40873_solr",
  "state":"active",
  "type":"NRT",
  "force_set_state":"false",
  "leader":"true"},
"core_node4":{
  "core":"collection1_shard1_replica_n2",
  "base_url":"http://127.0.0.1:33785/solr;,
  "node_name":"127.0.0.1:33785_solr",
  "state":"down",
  "type":"NRT",
  "force_set_state":"false",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
at 
__randomizedtesting.SeedInfo.seed([F52AFB9CDB19A1B5:7D7EC44675E5CC4D]:0)
at org.junit.Assert.fail(Assert.java:88)
at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:289)
at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:267)
at 
org.apache.solr.cloud.TestCloudRecovery2.test(TestCloudRecovery2.java:106)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at

[jira] [Commented] (SOLR-12313) TestInjection#waitForInSyncWithLeader needs improvement.

2019-01-24 Thread Hoss Man (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751838#comment-16751838
 ] 

Hoss Man commented on SOLR-12313:
-

Beyond the concerns mark previously raised regarding: documentation of purpose; 
how slow this can make tests; & wether it's correctly implemented – I've 
recently noticed two additional big problems:
 * this method causes *guaranteed* failures in 
{{ForceLeaderTest.testReplicasInLIRNoLeader}} on branch_7x when the randomizer 
enables java {{asserts}} _and_ {{onlyLeaderIndexes}} randomizes to "true" 
(resulting in the use of tlog replicas)
 ** Note: forcing {{TestInjection.waitForReplicasInSync = null}} allows this 
test to sometimes pass, but it's still finicky due to the delay in search 
results being updated in tlog replicas
 ** Mark had previously (Nov2018) modified this test class to prevent tlog 
replicas, but aparently didn't notice on backporting that 7x had this 
additional test method (deprecated & removed in master) that used the 
{{onlyLeaderIndexes}} variable directly
 * {{TestInjection.waitForInSyncWithLeader()}} does not obey the contract all 
test injection methods must obey
 ** notably it (by default) forces commits to tlog replicas to block until in 
sync with leader in non-test Solr instances if solr was started with JVM 
assertions
 ** this is an end user facing performance bottleneck, that can cause real 
world commits to time out!

Unless there are objections from someone who understands tlog replicas really 
well & is willing to step up and "fix" {{waitForInSyncWithLeader()}} (including 
mark's concerns above about how it's designed), I plan to do the following:
 # create a sub-task to document & track the user affecting bug regarding 
TLOG+assertions / commits in CHANGES.txt
 # commit a change making {{TestInjection.waitForInSyncWithLeader()}} a No-Op, 
citing both this issue and the new sub-task

 ** backporting down to 7x & resolving the sub-task as fixed in 7.7
 # commit a change to ForceLeaderTest's initialization of {{onlyLeaderIndexes}} 
completely preventing the use of tlog replicas as mark intended in his Nov2018 
commits
 ** backporting down to 7x

> TestInjection#waitForInSyncWithLeader needs improvement.
> 
>
> Key: SOLR-12313
> URL: https://issues.apache.org/jira/browse/SOLR-12313
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Priority: Major
>
> This really should have some doc for why it would be used.
> I also think it causes BasicDistributedZkTest to take forever for sometimes 
> and perhaps other tests?
> I think checking for uncommitted data is probably a race condition and should 
> be removed.
> Checking index versions should follow the rules that replication does - if 
> the slave is higher than the leader, it's in sync, being equal is not 
> required. If it's expected for a test it should be a specific test that 
> fails. This just introduces massive delays.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13125) Optimize Queries when sorting by router.field

2019-01-24 Thread Lucene/Solr QA (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751783#comment-16751783
 ] 

Lucene/Solr QA commented on SOLR-13125:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  1m 33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  1m 33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  1m 33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 40m 
38s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 29s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13125 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12956132/SOLR-13125.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon 
Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / c317119 |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | 1.8.0_191 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/267/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/267/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Optimize Queries when sorting by router.field
> -
>
> Key: SOLR-13125
> URL: https://issues.apache.org/jira/browse/SOLR-13125
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Minor
> Attachments: SOLR-13125-no-commit.patch, SOLR-13125.patch, 
> SOLR-13125.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We are currently testing TRA using Solr 7.7, having >300 shards in the alias, 
> with much growth in the coming months.
> The "hot" data(in our case, more recent) will be stored on stronger 
> nodes(SSD, more RAM, etc).
> A proposal of optimizing queries sorted by router.field(the field which TRA 
> uses to route the data to the correct collection) has emerged.
> Perhaps, in queries which are sorted by router.field, Solr could be smart 
> enough to wait for the more recent collections, and in case the limit was 
> reached cancel other queries(or just not block and wait for the results)?
> For example:
> When querying a TRA which with a filter on a different field than 
> router.field, but sorting by router.field desc, limit=100.
> Since this is a TRA, solr will issue queries for all the collections in the 
> alias.
> But to optimize this particular type of query, Solr could wait for the most 
> recent collection in the TRA, see whether the result set matches or exceeds 
> the limit. If so, the query could be returned to the user without waiting for 
> the rest of the shards. If not, the issuing node will block until the second 
> query returns, and so forth, until the limit of the request is reached.
> This might also be useful for deep paging, querying each collection and only 
> skipping to the next once there are no more results in the specified 
> collection.
> Thoughts or inputs are always welcome.
> This is just my two cents, and I'm always happy to brainstorm.
> Thanks in advance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (SOLR-12373) DocBasedVersionConstraintsProcessor doesn't work when schema has required fields

2019-01-24 Thread JIRA



 [ 
https://issues.apache.org/jira/browse/SOLR-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe updated SOLR-12373:
-
Attachment: SOLR-12373.patch

> DocBasedVersionConstraintsProcessor doesn't work when schema has required 
> fields
> 
>
> Key: SOLR-12373
> URL: https://issues.apache.org/jira/browse/SOLR-12373
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Tomás Fernández Löbbe
>Assignee: Tomás Fernández Löbbe
>Priority: Minor
> Attachments: SOLR-12373.patch, SOLR-12373.patch, SOLR-12373.patch, 
> SOLR-12373.patch
>
>
> DocBasedVersionConstraintsProcessor creates tombstones when processing a 
> delete by id. Those tombstones only have id (or whatever the unique key name 
> is) and version field(s), however, if the schema defines some required 
> fields, adding the tombstone will fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12373) DocBasedVersionConstraintsProcessor doesn't work when schema has required fields

2019-01-24 Thread JIRA



[ 
https://issues.apache.org/jira/browse/SOLR-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751743#comment-16751743
 ] 

Tomás Fernández Löbbe commented on SOLR-12373:
--

Attached a patch with the suggested change and some more tests. I'll commit 
soon.

> DocBasedVersionConstraintsProcessor doesn't work when schema has required 
> fields
> 
>
> Key: SOLR-12373
> URL: https://issues.apache.org/jira/browse/SOLR-12373
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Tomás Fernández Löbbe
>Assignee: Tomás Fernández Löbbe
>Priority: Minor
> Attachments: SOLR-12373.patch, SOLR-12373.patch, SOLR-12373.patch, 
> SOLR-12373.patch
>
>
> DocBasedVersionConstraintsProcessor creates tombstones when processing a 
> delete by id. Those tombstones only have id (or whatever the unique key name 
> is) and version field(s), however, if the schema defines some required 
> fields, adding the tombstone will fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1758 - Still Unstable

2019-01-24 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1758/

5 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [SolrZkClient, 
ZkStateReader] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.common.cloud.SolrZkClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:203)  
at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:126)  at 
org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:116)  at 
org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:306)  at 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:160)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:399)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
  at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)  at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224)  at 
org.apache.solr.client.solrj.impl.SolrClientCloudManager.request(SolrClientCloudManager.java:115)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:132)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:85)
  at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:324)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.common.cloud.ZkStateReader  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:328)  
at 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:160)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:399)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
  at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)  at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224)  at 
org.apache.solr.client.solrj.impl.SolrClientCloudManager.request(SolrClientCloudManager.java:115)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:132)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:85)
  at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:324)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)   expected null, but 
was:(SolrZkClient.java:203)  
at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:126)  at 
org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:116)  at 
org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:306)  at 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:160)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:399)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
  at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)  at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224)  at 
org.apache.solr.client.solrj.impl.SolrClientCloudManager.request(SolrClientCloudManager.java:115)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:132)
  at

[JENKINS] Lucene-Solr-repro - Build # 2726 - Unstable

2019-01-24 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-repro/2726/

[...truncated 53 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-Tests-master/3155/consoleText

[repro] Revision: 72a99e9c5c4f4f7ebcfa63c5cd386d48a9a8da3c

[repro] Repro line:  ant test  -Dtestcase=TestPrepRecovery 
-Dtests.method=testLeaderUnloaded -Dtests.seed=49AF66F46CA26568 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=tr-TR 
-Dtests.timezone=BET -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[repro] Repro line:  ant test  -Dtestcase=TestSimTriggerIntegration 
-Dtests.method=testSearchRate -Dtests.seed=49AF66F46CA26568 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sr-Latn 
-Dtests.timezone=SystemV/AST4 -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[repro] Repro line:  ant test  -Dtestcase=JWTAuthPluginIntegrationTest 
-Dtests.method=testMetrics -Dtests.seed=49AF66F46CA26568 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=es-CU -Dtests.timezone=America/Indianapolis 
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
c317119654c600f89eb29a423c51a8029f84033d
[repro] git fetch
[repro] git checkout 72a99e9c5c4f4f7ebcfa63c5cd386d48a9a8da3c

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   JWTAuthPluginIntegrationTest
[repro]   TestPrepRecovery
[repro]   TestSimTriggerIntegration
[repro] ant compile-test

[...truncated 3568 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 
-Dtests.class="*.JWTAuthPluginIntegrationTest|*.TestPrepRecovery|*.TestSimTriggerIntegration"
 -Dtests.showOutput=onerror  -Dtests.seed=49AF66F46CA26568 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=es-CU -Dtests.timezone=America/Indianapolis 
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[...truncated 2986 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.TestPrepRecovery
[repro]   0/5 failed: 
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration
[repro]   3/5 failed: org.apache.solr.security.JWTAuthPluginIntegrationTest
[repro] git checkout c317119654c600f89eb29a423c51a8029f84033d

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [DISCUSS] Opening old indices for reading

2019-01-24 Thread Michael Sokolov

+1 it makes sense to me; real world problems sometimes require messy
solutions. I guess the alternative is everybody develops their own suite of
tools and it is hard to share.

Some caution is warranted though I think; even with misc/experimental
caveats, these tools will only be useful if people can understand what to
expect from them, so it should be explicit what guarantees can be offered:
I don't know what they will be exactly, but supposing stored fields/doc
values fields can be retrieved/iterated over, but search results might
differ due to ranking differences, early termination relying on new index
structures?  Maybe naming/defining these as having a limited scope like
disaster recovery or migration or similar would give a hint that it should
not be used as some kind of adapter in a production system for old indexes.
I expect explaining what these tools are for to a wider audience will
deserve some care.

-Mike

On Wed, Jan 23, 2019 at 3:30 PM Erick Erickson 
wrote:

> +1, A lot of this was discussed on SOLR-12259, we should probably link
> any Lucene JIRAs for this back to that one to make an easy trail to
> follow.
>
> One thing I'd thought of is whether we should merge segments during
> this operation. If we're going to rewrite the entire index anyway,
> does it make sense to combine segments into max-sized segments a-la
> TieredMergePolicy?
>
> I'm not thinking of anything fancy at all here, there's no "cost" to
> calculate for instance. Just
> 1> go through the list of segments adding to a OneMerge until it's as
> big as it can be.
> 2> repeat until you have a list of OneMerge's that contain all the
> original segments.
>
> How big "as big as it can be" is TBD, TMP uses 5G. Could be a param I
> suppose.
>
> Erick
>
>
> On Wed, Jan 23, 2019 at 9:24 AM Andrzej Białecki  wrote:
> >
> > +1. I think that even with these caveats (read-only, some data may
> require re-interpretation) it would still be a great help for accessing
> legacy data, for which the original source may no longer exist.
> >
> > > On 23 Jan 2019, at 15:11, Simon Willnauer 
> wrote:
> > >
> > > Hey folks,
> > >
> > > tl;dr; I want to be able to open an indexreader on an old index if the
> > > SegmentInfo version is supported and all segment codecs are available.
> > > Today that's not possible even if I port old formats to current
> > > versions.
> > >
> > > Our BWC policy for quite a while has been N-1 major versions. That's
> > > good and I think we should keep it that way. Only recently, caused by
> > > changes how we encode/decode norms we also hard-enforce a the
> > > index-version-created in several places and the version a segment was
> > > written with. These are great enforcements and I understand why. My
> > > request here is if we can find consensus on allowing somehow (a
> > > special DirectoryReader for instance) to open such an index for
> > > reading only that doesn't provide the guarantees that our high level
> > > APIs decode norms correctly for instance. This would be enough to for
> > > instance consume stored fields etc. for reindexing or if a users are
> > > aware do they norms decoding in the codec. I am happy to work on a
> > > proposal how this would work. It would still enforce no writing or
> > > anything like this. I am also all for putting such a reader into misc
> > > and being experimental.
> > >
> > > simon
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Commented] (LUCENE-8658) Illegal assertion in WANDScorer

2019-01-24 Thread Jim Ferenczi (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751485#comment-16751485
 ] 

Jim Ferenczi commented on LUCENE-8658:
--

+1, thanks for fixing Adrien!

> Illegal assertion in WANDScorer
> ---
>
> Key: LUCENE-8658
> URL: https://issues.apache.org/jira/browse/LUCENE-8658
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8658.patch
>
>
> [~jim.ferenczi] told me about an assertion error that he ran into while 
> playing with WANDScorer.
> WANDScorer tries to not have to deal with accuracy issues on floating-point 
> numbers. In order to do this, it turns all scores into integers by 
> multiplying them by a scaling factor, and then rounding minimum competitive 
> scores down and rounding maximum scores up. This scaling factor is computed 
> in the constructor in such a way that scores end up in the 0..65536 range. 
> Sub scorers that have a maximum score of +Infty are ignored.
> The assertion is triggered in the rare case that a Scorer returns +Infty for 
> its maximum score when computing the scaling factor but then returns finite 
> values that are greater than the maximum scores of other clauses when asked 
> for the maximum score over smaller ranges of doc ids.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8658) Illegal assertion in WANDScorer

2019-01-24 Thread Adrien Grand (JIRA)

Adrien Grand created LUCENE-8658:


 Summary: Illegal assertion in WANDScorer
 Key: LUCENE-8658
 URL: https://issues.apache.org/jira/browse/LUCENE-8658
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Adrien Grand


[~jim.ferenczi] told me about an assertion error that he ran into while playing 
with WANDScorer.

WANDScorer tries to not have to deal with accuracy issues on floating-point 
numbers. In order to do this, it turns all scores into integers by multiplying 
them by a scaling factor, and then rounding minimum competitive scores down and 
rounding maximum scores up. This scaling factor is computed in the constructor 
in such a way that scores end up in the 0..65536 range. Sub scorers that have a 
maximum score of +Infty are ignored.

The assertion is triggered in the rare case that a Scorer returns +Infty for 
its maximum score when computing the scaling factor but then returns finite 
values that are greater than the maximum scores of other clauses when asked for 
the maximum score over smaller ranges of doc ids.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 2722 - Unstable

2019-01-24 Thread Apache Jenkins Server

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 23, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13156) Limiting field facet with certain terms via {!terms} not taking into account sorting

2019-01-24 Thread Andrey Kudryavtsev (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751404#comment-16751404
 ] 

Andrey Kudryavtsev commented on SOLR-13156:
---

{code:java}
protected NamedList getListedTermCounts(String field, final 
ParsedParams parsed, List terms) throws IOException {
  final SolrParams params = parsed.params;
  Integer mincount = parsed.params.getFieldInt(field, 
FacetParams.FACET_MINCOUNT);
  int offset = params.getFieldInt(field, FacetParams.FACET_OFFSET, 0);
  int limit = params.getFieldInt(field, FacetParams.FACET_LIMIT, 100);
  if (limit == 0) return new NamedList<>();
  if (mincount==null) {
Boolean zeros = params.getFieldBool(field, FacetParams.FACET_ZEROS);
mincount = (zeros!=null && !zeros) ? 1 : 0;
  }
  boolean missing = params.getFieldBool(field, FacetParams.FACET_MISSING, 
false);
  String sort = params.getFieldParam(field, FacetParams.FACET_SORT);
  String prefix = params.getFieldParam(field, FacetParams.FACET_PREFIX);
  final HashSet includeTerms = new HashSet<>(terms);
  final Predicate termFilter = bytesRef -> 
includeTerms.contains(bytesRef.utf8ToString());
  boolean exists = params.getFieldBool(field, FacetParams.FACET_EXISTS, false);

  return getFacetTermEnumCounts(searcher, parsed.docs, field, offset, limit, 
mincount, missing, sort, prefix, termFilter, exists);
}{code}
 

"List terms order' doesn't work obviously, but 'code reusing' level is high. 

> Limiting field facet with certain terms via {!terms} not taking into account 
> sorting
> 
>
> Key: SOLR-13156
> URL: https://issues.apache.org/jira/browse/SOLR-13156
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Konstantin Perikov
>Priority: Major
> Attachments: SOLR-13156.patch
>
>
> When I'm doing limiting facet keys with \{!terms} it doesn't take into 
> account sorting.
> First query not limiting the facet keys:
> {{facet.field=title=count=on=*:*}}
> Response as expected:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "book2",3, "book1",2, "book3",1]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
>  
> When doing it with limiting:
> {{facet.field=\{!terms=Book3,Book2,Book1}title=count=on=*:*}}
> I'm getting the exact order of how I list terms:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "Book3",1, "Book2",3, "Book1",2]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
> I've looked at the code, and it's clearly an issue there:
>  
> org.apache.solr.request.SimpleFacets#getListedTermCounts
>  
> {{for (String term : terms) {}}
> {{    int count = searcher.numDocs(ft.getFieldQuery(null, sf, term), 
> parsed.docs);}}
> {{    res.add(term, count);}}
> {{}}}
>  
> it's just basically iterating over terms and don't do any sorting at all. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Lucene/Solr 8.0

2019-01-24 Thread Uwe Schindler

Cool,

I am working on giving my best release time guess as possible on the FOSDEM 
conference!

Uwe

-
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

> -Original Message-
> From: Adrien Grand 
> Sent: Thursday, January 24, 2019 5:33 PM
> To: Lucene Dev 
> Subject: Re: Lucene/Solr 8.0
> 
> +1 to release 7.7 and 8.0 in a row starting on the week of February 4th.
> 
> On Wed, Jan 23, 2019 at 4:23 PM jim ferenczi 
> wrote:
> >
> > Hi,
> > As we agreed some time ago I'd like to start on releasing 8.0. The branch is
> already created so we can start the process anytime now. Unless there are
> objections I'd like to start the feature freeze next week in order to build 
> the
> first candidate the week after.
> > We'll also need a 7.7 release but I think we can handle both with Alan so
> the question now is whether we are ok to start the release process or if there
> are any blockers left ;).
> >
> >
> > Le mar. 15 janv. 2019 à 11:35, Alan Woodward 
> a écrit :
> >>
> >> I’ve started to work through the various deprecations on the new master
> branch.  There are a lot of them, and I’m going to need some assistance for
> several of them, as it’s not entirely clear what to do.
> >>
> >> I’ll open two overarching issues in JIRA, one for lucene and one for Solr,
> with lists of the deprecations that need to be removed in each one.  I’ll 
> create
> a shared branch on gitbox to work against, and push the changes I’ve already
> done there.  We can then create individual JIRA issues for any changes that
> are more involved than just deleting code.
> >>
> >> All assistance gratefully received, particularly for the Solr deprecations
> where there’s a lot of code I’m unfamiliar with.
> >>
> >> On 8 Jan 2019, at 09:21, Alan Woodward 
> wrote:
> >>
> >> I think the current plan is to do a 7.7 release at the same time as 8.0, to
> handle any last-minute deprecations etc.  So let’s keep those jobs enabled
> for now.
> >>
> >> On 8 Jan 2019, at 09:10, Uwe Schindler  wrote:
> >>
> >> Hi,
> >>
> >> I will start and add the branch_8x jobs to Jenkins once I have some time
> later today.
> >>
> >> The question: How to proceed with branch_7x? Should we stop using it
> and release 7.6.x only (so we would use branch_7_6 only for bugfixes), or
> are we planning to one more Lucene/Solr 7.7? In the latter case I would keep
> the jenkins jobs enabled for a while.
> >>
> >> Uwe
> >>
> >> -
> >> Uwe Schindler
> >> Achterdiek 19, D-28357 Bremen
> >> http://www.thetaphi.de
> >> eMail: u...@thetaphi.de
> >>
> >> From: Alan Woodward 
> >> Sent: Monday, January 7, 2019 11:30 AM
> >> To: dev@lucene.apache.org
> >> Subject: Re: Lucene/Solr 8.0
> >>
> >> OK, Christmas caught up with me a bit… I’ve just created a branch for 8x
> from master, and am in the process of updating the master branch to version
> 9.  New commits that should be included in the 8.0 release should also be
> back-ported to branch_8x from master.
> >>
> >> This is not intended as a feature freeze, as I know there are still some
> things being worked on for 8.0; however, it should let us clean up master by
> removing as much deprecated code as possible, and give us an idea of any
> replacement work that needs to be done.
> >>
> >>
> >> On 19 Dec 2018, at 15:13, David Smiley 
> wrote:
> >>
> >> January.
> >>
> >> On Wed, Dec 19, 2018 at 2:04 AM S G 
> wrote:
> >>
> >> It would be nice to see Solr 8 in January soon as there is an enhancement
> on nested-documents we are waiting to get our hands on.
> >> Any idea when Solr 8 would be out ?
> >>
> >> Thx
> >> SG
> >>
> >> On Mon, Dec 17, 2018 at 1:34 PM David Smiley
>  wrote:
> >>
> >> I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND
> priority = Blocker and status = open and fixVersion = "master (8.0)"
> >>click here:
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(SOLR%2C%20LU
> CENE)%20AND%20priority%20%3D%20Blocker%20and%20status%20%3D%2
> 0open%20and%20fixVersion%20%3D%20%22master%20(8.0)%22%20
> >>
> >> Thru the end of the month, I intend to work on those issues not yet
> assigned.
> >>
> >> On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand 
> wrote:
> >>
> >> +1
> >>
> >> On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward
>  wrote:
> >> >
> >> > Hi all,
> >> >
> >> > Now that 7.6 is out of the door (thanks Nick!) we should think about
> cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create the
> branch this week - say Wednesday?  Then we should have some time to
> clean up the master branch and uncover anything that still needs to be done
> on 8.0 before we start the release process next year.
> >> >
> >> > On 22 Oct 2018, at 18:12, Cassandra Targett 
> wrote:
> >> >
> >> > I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
> >> >
> >> > On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson
>  wrote:
> >> >>
> >> >> +1, this gives us all a chance to prioritize getting the blockers out
> >> >> of the way

[jira] [Commented] (SOLR-13165) enabling docValues on a tdate field and searching on the field is very slow

2019-01-24 Thread Erick Erickson (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751364#comment-16751364
 ] 

Erick Erickson commented on SOLR-13165:
---

Also, you _must_ reindex all documents from scratch when you make changes like 
this to get valid results. If at all possible, try with a new _collection_.

docValues="true" indexed="false" is analogous to a "table scan" in database 
terms. As Yonik says, docValues="true" indexed="true" should be fast.

> enabling docValues on a tdate field and searching on the field is very slow
> ---
>
> Key: SOLR-13165
> URL: https://issues.apache.org/jira/browse/SOLR-13165
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Sheeba Dhanaraj
>Priority: Major
>
> when we enable docValues on a tdate field and search on the field response 
> time is very slow. when we remove docValues from the field performance is 
> significantly improved. Is this by design? should we not enable docValues for 
> tdate fields



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13165) enabling docValues on a tdate field and searching on the field is very slow

2019-01-24 Thread Sheeba Dhanaraj (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751372#comment-16751372
 ] 

Sheeba Dhanaraj commented on SOLR-13165:


yes I'm reindexing when I add/remove docValues

when I search on the test field search query response is slow when docValues is 
enabled on this field

  fast

  slow

> enabling docValues on a tdate field and searching on the field is very slow
> ---
>
> Key: SOLR-13165
> URL: https://issues.apache.org/jira/browse/SOLR-13165
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Sheeba Dhanaraj
>Priority: Major
>
> when we enable docValues on a tdate field and search on the field response 
> time is very slow. when we remove docValues from the field performance is 
> significantly improved. Is this by design? should we not enable docValues for 
> tdate fields



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13165) enabling docValues on a tdate field and searching on the field is very slow

2019-01-24 Thread Yonik Seeley (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751303#comment-16751303
 ] 

Yonik Seeley commented on SOLR-13165:
-

Are you sure that the field was indexed both times?
As long as the tdate field is indexed, that index should be used for queries, 
regardless of if it has docValues.

> enabling docValues on a tdate field and searching on the field is very slow
> ---
>
> Key: SOLR-13165
> URL: https://issues.apache.org/jira/browse/SOLR-13165
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Sheeba Dhanaraj
>Priority: Major
>
> when we enable docValues on a tdate field and search on the field response 
> time is very slow. when we remove docValues from the field performance is 
> significantly improved. Is this by design? should we not enable docValues for 
> tdate fields



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-13165) enabling docValues on a tdate field and searching on the field is very slow

2019-01-24 Thread Sheeba Dhanaraj (JIRA)

Sheeba Dhanaraj created SOLR-13165:
--

 Summary: enabling docValues on a tdate field and searching on the 
field is very slow
 Key: SOLR-13165
 URL: https://issues.apache.org/jira/browse/SOLR-13165
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Sheeba Dhanaraj


when we enable docValues on a tdate field and search on the field response time 
is very slow. when we remove docValues from the field performance is 
significantly improved. Is this by design? should we not enable docValues for 
tdate fields



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 8.0

2019-01-24 Thread Adrien Grand

+1 to release 7.7 and 8.0 in a row starting on the week of February 4th.

On Wed, Jan 23, 2019 at 4:23 PM jim ferenczi  wrote:
>
> Hi,
> As we agreed some time ago I'd like to start on releasing 8.0. The branch is 
> already created so we can start the process anytime now. Unless there are 
> objections I'd like to start the feature freeze next week in order to build 
> the first candidate the week after.
> We'll also need a 7.7 release but I think we can handle both with Alan so the 
> question now is whether we are ok to start the release process or if there 
> are any blockers left ;).
>
>
> Le mar. 15 janv. 2019 à 11:35, Alan Woodward  a écrit :
>>
>> I’ve started to work through the various deprecations on the new master 
>> branch.  There are a lot of them, and I’m going to need some assistance for 
>> several of them, as it’s not entirely clear what to do.
>>
>> I’ll open two overarching issues in JIRA, one for lucene and one for Solr, 
>> with lists of the deprecations that need to be removed in each one.  I’ll 
>> create a shared branch on gitbox to work against, and push the changes I’ve 
>> already done there.  We can then create individual JIRA issues for any 
>> changes that are more involved than just deleting code.
>>
>> All assistance gratefully received, particularly for the Solr deprecations 
>> where there’s a lot of code I’m unfamiliar with.
>>
>> On 8 Jan 2019, at 09:21, Alan Woodward  wrote:
>>
>> I think the current plan is to do a 7.7 release at the same time as 8.0, to 
>> handle any last-minute deprecations etc.  So let’s keep those jobs enabled 
>> for now.
>>
>> On 8 Jan 2019, at 09:10, Uwe Schindler  wrote:
>>
>> Hi,
>>
>> I will start and add the branch_8x jobs to Jenkins once I have some time 
>> later today.
>>
>> The question: How to proceed with branch_7x? Should we stop using it and 
>> release 7.6.x only (so we would use branch_7_6 only for bugfixes), or are we 
>> planning to one more Lucene/Solr 7.7? In the latter case I would keep the 
>> jenkins jobs enabled for a while.
>>
>> Uwe
>>
>> -
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> http://www.thetaphi.de
>> eMail: u...@thetaphi.de
>>
>> From: Alan Woodward 
>> Sent: Monday, January 7, 2019 11:30 AM
>> To: dev@lucene.apache.org
>> Subject: Re: Lucene/Solr 8.0
>>
>> OK, Christmas caught up with me a bit… I’ve just created a branch for 8x 
>> from master, and am in the process of updating the master branch to version 
>> 9.  New commits that should be included in the 8.0 release should also be 
>> back-ported to branch_8x from master.
>>
>> This is not intended as a feature freeze, as I know there are still some 
>> things being worked on for 8.0; however, it should let us clean up master by 
>> removing as much deprecated code as possible, and give us an idea of any 
>> replacement work that needs to be done.
>>
>>
>> On 19 Dec 2018, at 15:13, David Smiley  wrote:
>>
>> January.
>>
>> On Wed, Dec 19, 2018 at 2:04 AM S G  wrote:
>>
>> It would be nice to see Solr 8 in January soon as there is an enhancement on 
>> nested-documents we are waiting to get our hands on.
>> Any idea when Solr 8 would be out ?
>>
>> Thx
>> SG
>>
>> On Mon, Dec 17, 2018 at 1:34 PM David Smiley  
>> wrote:
>>
>> I see 10 JIRA issues matching this filter:   project in (SOLR, LUCENE) AND 
>> priority = Blocker and status = open and fixVersion = "master (8.0)"
>>click here:
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(SOLR%2C%20LUCENE)%20AND%20priority%20%3D%20Blocker%20and%20status%20%3D%20open%20and%20fixVersion%20%3D%20%22master%20(8.0)%22%20
>>
>> Thru the end of the month, I intend to work on those issues not yet assigned.
>>
>> On Mon, Dec 17, 2018 at 4:51 AM Adrien Grand  wrote:
>>
>> +1
>>
>> On Mon, Dec 17, 2018 at 10:38 AM Alan Woodward  wrote:
>> >
>> > Hi all,
>> >
>> > Now that 7.6 is out of the door (thanks Nick!) we should think about 
>> > cutting the 8.0 branch and moving master to 9.0.  I’ll volunteer to create 
>> > the branch this week - say Wednesday?  Then we should have some time to 
>> > clean up the master branch and uncover anything that still needs to be 
>> > done on 8.0 before we start the release process next year.
>> >
>> > On 22 Oct 2018, at 18:12, Cassandra Targett  wrote:
>> >
>> > I'm a bit delayed, but +1 on the 7.6 and 8.0 plan from me too.
>> >
>> > On Fri, Oct 19, 2018 at 7:18 AM Erick Erickson  
>> > wrote:
>> >>
>> >> +1, this gives us all a chance to prioritize getting the blockers out
>> >> of the way in a careful manner.
>> >> On Fri, Oct 19, 2018 at 7:56 AM jim ferenczi  
>> >> wrote:
>> >> >
>> >> > +1 too. With this new perspective we could create the branch just after 
>> >> > the 7.6 release and target the 8.0 release for January 2019 which gives 
>> >> > almost 3 month to finish the blockers ?
>> >> >
>> >> > Le jeu. 18 oct. 2018 à 23:56, David Smiley  a 
>> >> > écrit :
>> >> >>
>> >> >> +1 to a 7.6 —lots of stuff in there
>> >> >> On Thu, Oct 18, 2018 at 4:47 PM

[jira] [Commented] (SOLR-13029) Allow HDFS backup/restore buffer size to be configured

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751281#comment-16751281
 ] 

ASF subversion and git services commented on SOLR-13029:


Commit 5a54c624cac4d793c7e65c980eccde3c2d63e61b in lucene-solr's branch 
refs/heads/branch_8x from Mikhail Khludnev
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5a54c62 ]

SOLR-13029: configure buffer size in HdfsBackupRepository.


> Allow HDFS backup/restore buffer size to be configured
> --
>
> Key: SOLR-13029
> URL: https://issues.apache.org/jira/browse/SOLR-13029
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, hdfs
>Affects Versions: 7.5, 8.0
>Reporter: Tim Owen
>Priority: Major
> Attachments: SOLR-13029.patch, SOLR-13029.patch, SOLR-13029.patch
>
>
> There's a default hardcoded buffer size setting of 4096 in the HDFS code 
> which means in particular that restoring a backup from HDFS takes a long 
> time. Copying multi-GB files from HDFS using a buffer as small as 4096 bytes 
> is very inefficient. We changed this in our local build used in production to 
> 256kB and saw a 10x speed improvement when restoring a backup. Attached patch 
> simply makes this size configurable using a command line setting, much like 
> several other buffer size values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13029) Allow HDFS backup/restore buffer size to be configured

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751280#comment-16751280
 ] 

ASF subversion and git services commented on SOLR-13029:


Commit c1ce24e0cfbd40aacc3c0f0117d9be56cc1b7936 in lucene-solr's branch 
refs/heads/branch_7x from Mikhail Khludnev
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c1ce24e ]

SOLR-13029: configure buffer size in HdfsBackupRepository.


> Allow HDFS backup/restore buffer size to be configured
> --
>
> Key: SOLR-13029
> URL: https://issues.apache.org/jira/browse/SOLR-13029
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, hdfs
>Affects Versions: 7.5, 8.0
>Reporter: Tim Owen
>Priority: Major
> Attachments: SOLR-13029.patch, SOLR-13029.patch, SOLR-13029.patch
>
>
> There's a default hardcoded buffer size setting of 4096 in the HDFS code 
> which means in particular that restoring a backup from HDFS takes a long 
> time. Copying multi-GB files from HDFS using a buffer as small as 4096 bytes 
> is very inefficient. We changed this in our local build used in production to 
> 256kB and saw a 10x speed improvement when restoring a backup. Attached patch 
> simply makes this size configurable using a command line setting, much like 
> several other buffer size values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13029) Allow HDFS backup/restore buffer size to be configured

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751270#comment-16751270
 ] 

ASF subversion and git services commented on SOLR-13029:


Commit c317119654c600f89eb29a423c51a8029f84033d in lucene-solr's branch 
refs/heads/master from Mikhail Khludnev
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c317119 ]

SOLR-13029: configure buffer size in HdfsBackupRepository.


> Allow HDFS backup/restore buffer size to be configured
> --
>
> Key: SOLR-13029
> URL: https://issues.apache.org/jira/browse/SOLR-13029
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, hdfs
>Affects Versions: 7.5, 8.0
>Reporter: Tim Owen
>Priority: Major
> Attachments: SOLR-13029.patch, SOLR-13029.patch, SOLR-13029.patch
>
>
> There's a default hardcoded buffer size setting of 4096 in the HDFS code 
> which means in particular that restoring a backup from HDFS takes a long 
> time. Copying multi-GB files from HDFS using a buffer as small as 4096 bytes 
> is very inefficient. We changed this in our local build used in production to 
> 256kB and saw a 10x speed improvement when restoring a backup. Attached patch 
> simply makes this size configurable using a command line setting, much like 
> several other buffer size values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Size Limitation upto 32 kb limitation

2019-01-24 Thread Erick Erickson

What Jan said.

If you are getting this error on a _text_ based field, then your data
is bad. What it's telling you is that _after_ tokenization, you have a
single _term_ that's > 32K which is almost, but not quite totally,
useless.

ImagineASingleWordThatRunsOnForMoreThanThirtyTwoThousandCharactersHowWouldThatBeUsefulToEitherSearchOrReturnToTheUserWouldThisSingleWordImTypingBeUsefulAndItIsntEvenCloseToThirtyTwoThousandCharacters.

So I'd try to find out what it is you're processing that shows you
such a large term. It's pretty easy to run Tika on a file in SolrJ,
see: https://lucidworks.com/2012/02/14/indexing-with-solrj/

There are also web sites that'll process the PDF file through Tika and
show you how it parses

Best,
Erick

On Thu, Jan 24, 2019 at 12:57 AM Jan Høydahl  wrote:
>
> I cannot see why you'd want a single term of 32kb in your index anyway. Can 
> you give us examples of what these terms are and how you will search them?
> What kind of files are you indexing, could it be like bad PDFs consisting of 
> a bunch of binary garbage?
> Try adding a lengthFilterFactory to your fieldType(s). See 
> https://lucene.apache.org/solr/guide/7_6/filter-descriptions.html#length-filter
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> 24. jan. 2019 kl. 06:51 skrev Kranthi Kumar K 
> :
>
> Thank you Bernd Fehling for your suggested solution, I've tried the same by 
> changing the type and added multivalued to true in Schema.xml file i.e,
> change from:
>
> 
>
> Changed to:
>
>  multiValued="true" />
>
> After changing it also still we are unable to import the files size > 32 kb. 
> please find the solution suggested by Bernd in the below url:
>
> http://lucene.472066.n3.nabble.com/Re-Solr-Size-Limitation-upto-32-kb-limitation-td4421569.html
>
> Bernd Fehling, could you please suggest another alternative solution to 
> resolve our issue, which would help us alot?
>
> Please let me know for any questions.
>
>
> 
>
> Thanks & Regards,
> Kranthi Kumar.K,
> Software Engineer,
> Ccube Fintech Global Services Pvt Ltd.,
> Email/Skype: kranthikuma...@ccubefintech.com,
> Mobile: +91-8978078449.
>
>
> From: Kranthi Kumar K
> Sent: Friday, January 18, 2019 4:22 PM
> To: dev@lucene.apache.org; solr-u...@lucene.apache.org
> Cc: Ananda Babu medida ; Srinivasa Reddy 
> Karri ; Michelle Ngo 
> ; Ravi Vangala 
> Subject: RE: Solr Size Limitation upto 32 kb limitation
>
> Hi team,
>
> Thank you Erick Erickson ,Bernd Fehling , Jan Hoydahl for your suggested 
> solutions. I’ve tried the suggested one’s and still we are unable to import 
> files havingsize  >32 kb, it is displaying same error.
>
> Below link has the suggested solutions. Please have a look once.
>
> http://lucene.472066.n3.nabble.com/Solr-Size-Limitation-upto-32-KB-files-td4419779.html
>
>
> As per Erick Erickson, I’ve changed the string type to Text type based and 
> still the issue occurs .
>
> I’ve changed from :
>
> 
>
> Changed to:
>
> 
>
> If we do so, it is showing error in the log, please find the error in the 
> attachment.
>
> If I change to:
>
> 
>
> It is not showing any error , but the issue still exists.
>
>
> As per Jan Hoydahl, I have gone through the link that you have provided and 
> checked ‘requestParsers’ tag in solrconfig.xml,
>
>
> RequestParsers tag in our application is as follows:
>
> ‘ multipartUploadLimitInKB="2048000"
> formdataUploadLimitInKB="2048"
> addHttpRequestToContext="false"/>’
> Request parsers, which we are using and in the link you have provided are 
> similar. And still we are unable to import the files size >32 kb.
>
>
> As per Bernd Fehling, we are using Solr 4.10.2. you have mentioned as,
>
> ‘If you are trying to add larger content then you have to "chop" that
> by yourself and add it as multivalued. Can be done within a self written 
> loader. ’
>
> I’m a newbie to Solr and I didn’t get what exactly ‘self written loader’ is?
>
> Could you please provide us sample code, that helps us to go further?
>
>
>
> 
>
> Thanks & Regards,
> Kranthi Kumar.K,
> Software Engineer,
> Ccube Fintech Global Services Pvt Ltd.,
> Email/Skype: kranthikuma...@ccubefintech.com,
> Mobile: +91-8978078449.
>
>
> From: Kranthi Kumar K 
> Sent: Thursday, January 17, 2019 12:43 PM
> To: dev@lucene.apache.org; solr-u...@lucene.apache.org
> Cc: Ananda Babu medida ; Srinivasa Reddy 
> Karri ; Michelle Ngo 
> 
> Subject: Re: Solr Size Limitation upto 32 kb limitation
>
>
> Hi Team,
>
>
>
> Can we have any updates on the below issue? We are awaiting your reply.
>
>
>
> Thanks,
>
> Kranthi kumar.K
>
> 
> From: Kranthi Kumar K
> Sent: Friday, January 4, 2019 5:01:38 PM
> To: dev@lucene.apache.org
> Cc: Ananda Babu medida; Srinivasa Reddy Karri
> Subject: Solr Size Limitation upto 32 kb limitation
>
>
> Hi team,
>
>
>
> We are currently using Solr 4.2.1 version in our project and everything is 
> going

[JENKINS] Lucene-Solr-8.x-MacOSX (64bit/jdk1.8.0) - Build # 22 - Unstable!

2019-01-24 Thread Policeman Jenkins Server

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/22/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.ExecutePlanActionTest

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [SolrZkClient, 
ZkStateReader] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.common.cloud.SolrZkClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:203)  
at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:126)  at 
org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:116)  at 
org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:306)  at 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:160)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:399)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
  at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)  at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224)  at 
org.apache.solr.client.solrj.impl.SolrClientCloudManager.request(SolrClientCloudManager.java:115)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:132)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:85)
  at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:324)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.common.cloud.ZkStateReader  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:328)  
at 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:160)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:399)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
  at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)  at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224)  at 
org.apache.solr.client.solrj.impl.SolrClientCloudManager.request(SolrClientCloudManager.java:115)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:132)
  at 
org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:85)
  at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:324)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)   expected null, but 
was:(SolrZkClient.java:203)  
at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:126)  at 
org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:116)  at 
org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:306)  at 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:160)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:399)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
  at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
  at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)  at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224)  at 
org.apache.solr.client.solrj.impl.SolrClientCloudManager.request(SolrClientCloudManager.java:115)
  at

[jira] [Resolved] (SOLR-13164) solr 6.6.3 is not working on azure web app, it throws an error "The server committed a protocol violation. Section=ResponseStatusLine"

2019-01-24 Thread JIRA



 [ 
https://issues.apache.org/jira/browse/SOLR-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-13164.

Resolution: Invalid

Marking as invalid. Please do not use the project bug tracker as a support 
portal for asking questions about how to deploy Solr.

The correct way to get help is to subscribe to the solr-user mailing list and 
ask a question there 
(http://lucene.apache.org/solr/community.html#mailing-lists-irc). When you do 
ask a question, make sure to provide much more details about your setup. I 
don't even know what you mean by "We have deployed solr 6.6.3 on azure as a web 
app". Solr is an application, not a web app!

> solr 6.6.3 is not working on azure web app, it throws an error "The server 
> committed a protocol violation. Section=ResponseStatusLine"
> --
>
> Key: SOLR-13164
> URL: https://issues.apache.org/jira/browse/SOLR-13164
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: website
>Affects Versions: 6.6.3
> Environment: solr 6.6.3
> sitecore 8.2 initial release
>Reporter: Jignesh Savaliya
>Priority: Blocker
> Fix For: 6.6.3
>
> Attachments: solr error.jpg
>
>
> We have deployed solr 6.6.3 on azure as a web app and it works fine over 
> there, but when we try to integrate the same with our sitecore(version 8.2) 
> which is deployed on azure vm, at the time of re-indexing it throws an error 
> of "The server committed a protocol violation. Section=ResponseStatusLine".
> It works fine if we deploy the solr 6.6.3 on azure vm. please suggest, thanks 
> in advance.
> Please refer attached SS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_172) - Build # 3447 - Unstable!

2019-01-24 Thread Policeman Jenkins Server

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3447/
Java: 32bit/jdk1.8.0_172 -client -XX:+UseG1GC

5 tests failed.
FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.test

Error Message:
Timeout waiting for new replica become leader Timeout waiting to see state for 
collection=collection1 
:DocCollection(collection1//collections/collection1/state.json/5)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node3":{   "core":"collection1_shard1_replica_n1",   
"base_url":"http://127.0.0.1:43703/solr;,   
"node_name":"127.0.0.1:43703_solr",   "state":"active",   
"type":"NRT",   "force_set_state":"false",   "leader":"true"},  
   "core_node4":{   "core":"collection1_shard1_replica_n2", 
  "base_url":"http://127.0.0.1:40905/solr;,   
"node_name":"127.0.0.1:40905_solr",   "state":"active",   
"type":"NRT",   "force_set_state":"false",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"} Live 
Nodes: [127.0.0.1:40905_solr, 127.0.0.1:43703_solr] Last available state: 
DocCollection(collection1//collections/collection1/state.json/5)={   
"pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node3":{   "core":"collection1_shard1_replica_n1",   
"base_url":"http://127.0.0.1:43703/solr;,   
"node_name":"127.0.0.1:43703_solr",   "state":"active",   
"type":"NRT",   "force_set_state":"false",   "leader":"true"},  
   "core_node4":{   "core":"collection1_shard1_replica_n2", 
  "base_url":"http://127.0.0.1:40905/solr;,   
"node_name":"127.0.0.1:40905_solr",   "state":"active",   
"type":"NRT",   "force_set_state":"false",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Timeout waiting for new replica become leader
Timeout waiting to see state for collection=collection1 
:DocCollection(collection1//collections/collection1/state.json/5)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node3":{
  "core":"collection1_shard1_replica_n1",
  "base_url":"http://127.0.0.1:43703/solr;,
  "node_name":"127.0.0.1:43703_solr",
  "state":"active",
  "type":"NRT",
  "force_set_state":"false",
  "leader":"true"},
"core_node4":{
  "core":"collection1_shard1_replica_n2",
  "base_url":"http://127.0.0.1:40905/solr;,
  "node_name":"127.0.0.1:40905_solr",
  "state":"active",
  "type":"NRT",
  "force_set_state":"false",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
Live Nodes: [127.0.0.1:40905_solr, 127.0.0.1:43703_solr]
Last available state: 
DocCollection(collection1//collections/collection1/state.json/5)={
  "pullReplicas":"0",
  "replicationFactor":"2",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node3":{
  "core":"collection1_shard1_replica_n1",
  "base_url":"http://127.0.0.1:43703/solr;,
  "node_name":"127.0.0.1:43703_solr",
  "state":"active",
  "type":"NRT",
  "force_set_state":"false",
  "leader":"true"},
"core_node4":{
  "core":"collection1_shard1_replica_n2",
  "base_url":"http://127.0.0.1:40905/solr;,
  "node_name":"127.0.0.1:40905_solr",
  "state":"active",
  "type":"NRT",
  "force_set_state":"false",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0"}
at 
__randomizedtesting.SeedInfo.seed([8448D61A79F83A3E:C1CE9C0D70457C6]:0)
at org.junit.Assert.fail(Assert.java:88)
at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:289)
at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:267)
at 
org.apache.solr.cloud.LeaderTragicEventTest.test(LeaderTragicEventTest.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at

[jira] [Created] (SOLR-13164) solr 6.6.3 is not working on azure web app, it throws an error "The server committed a protocol violation. Section=ResponseStatusLine"

2019-01-24 Thread Jignesh Savaliya (JIRA)

Jignesh Savaliya created SOLR-13164:
---

 Summary: solr 6.6.3 is not working on azure web app, it throws an 
error "The server committed a protocol violation. Section=ResponseStatusLine"
 Key: SOLR-13164
 URL: https://issues.apache.org/jira/browse/SOLR-13164
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: website
Affects Versions: 6.6.3
 Environment: solr 6.6.3

sitecore 8.2 initial release
Reporter: Jignesh Savaliya
 Fix For: 6.6.3
 Attachments: solr error.jpg

We have deployed solr 6.6.3 on azure as a web app and it works fine over there, 
but when we try to integrate the same with our sitecore(version 8.2) which is 
deployed on azure vm, at the time of re-indexing it throws an error of "The 
server committed a protocol violation. Section=ResponseStatusLine".

It works fine if we deploy the solr 6.6.3 on azure vm. please suggest, thanks 
in advance.

Please refer attached SS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 2724 - Unstable

2019-01-24 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-repro/2724/

[...truncated 29 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/268/consoleText

[repro] Revision: 72a99e9c5c4f4f7ebcfa63c5cd386d48a9a8da3c

[repro] Repro line:  ant test  -Dtestcase=MetricTriggerIntegrationTest 
-Dtests.method=testMetricTrigger -Dtests.seed=93DD85DA4A365311 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=it-IT -Dtests.timezone=Asia/Jerusalem -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestRequestForwarding 
-Dtests.method=testMultiCollectionQuery -Dtests.seed=93DD85DA4A365311 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=fr-LU -Dtests.timezone=Pacific/Yap -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=SolrExampleStreamingTest 
-Dtests.method=testCommitWithinOnDelete -Dtests.seed=6D4C3E501DCAFB89 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ga 
-Dtests.timezone=America/Monterrey -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
d09c724a0eaca3666dcb3fbc931bb64eb5f5b72f
[repro] git fetch
[repro] git checkout 72a99e9c5c4f4f7ebcfa63c5cd386d48a9a8da3c

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   TestRequestForwarding
[repro]   MetricTriggerIntegrationTest
[repro]solr/solrj
[repro]   SolrExampleStreamingTest
[repro] ant compile-test

[...truncated 3568 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 
-Dtests.class="*.TestRequestForwarding|*.MetricTriggerIntegrationTest" 
-Dtests.showOutput=onerror  -Dtests.seed=93DD85DA4A365311 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=fr-LU 
-Dtests.timezone=Pacific/Yap -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 2925 lines...]
[repro] Setting last failure code to 256

[repro] ant compile-test

[...truncated 454 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.SolrExampleStreamingTest" -Dtests.showOutput=onerror  
-Dtests.seed=6D4C3E501DCAFB89 -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=ga -Dtests.timezone=America/Monterrey 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 257 lines...]
[repro] Failures:
[repro]   0/5 failed: 
org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest
[repro]   0/5 failed: org.apache.solr.cloud.TestRequestForwarding
[repro]   2/5 failed: 
org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest
[repro] git checkout d09c724a0eaca3666dcb3fbc931bb64eb5f5b72f

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-7.x - Build # 1221 - Unstable

2019-01-24 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/1221/

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 3 object(s) that were not released!!! [SolrCore, 
InternalHttpClient, MockDirectoryWrapper] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.core.SolrCore  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.core.SolrCore.(SolrCore.java:1054)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:874)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
  at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:690)  
at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
  at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:225)  
at org.apache.solr.handler.IndexFetcher.(IndexFetcher.java:267)  at 
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1202) 
 at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:696) 
 at org.apache.solr.core.SolrCore.(SolrCore.java:1000)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:874)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
  at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:690)  
at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MockDirectoryWrapper  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:95)  at 
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:770)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:967)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:874)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
  at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:690)  
at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)   expected null, but 
was:(SolrCore.java:1054)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:874)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
  at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:690)  
at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at

[jira] [Updated] (SOLR-13156) Limiting field facet with certain terms via {!terms} not taking into account sorting

2019-01-24 Thread Konstantin Perikov (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Perikov updated SOLR-13156:
--
Attachment: (was: SOLR-13156.patch)

> Limiting field facet with certain terms via {!terms} not taking into account 
> sorting
> 
>
> Key: SOLR-13156
> URL: https://issues.apache.org/jira/browse/SOLR-13156
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Konstantin Perikov
>Priority: Major
> Attachments: SOLR-13156.patch
>
>
> When I'm doing limiting facet keys with \{!terms} it doesn't take into 
> account sorting.
> First query not limiting the facet keys:
> {{facet.field=title=count=on=*:*}}
> Response as expected:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "book2",3, "book1",2, "book3",1]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
>  
> When doing it with limiting:
> {{facet.field=\{!terms=Book3,Book2,Book1}title=count=on=*:*}}
> I'm getting the exact order of how I list terms:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "Book3",1, "Book2",3, "Book1",2]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
> I've looked at the code, and it's clearly an issue there:
>  
> org.apache.solr.request.SimpleFacets#getListedTermCounts
>  
> {{for (String term : terms) {}}
> {{    int count = searcher.numDocs(ft.getFieldQuery(null, sf, term), 
> parsed.docs);}}
> {{    res.add(term, count);}}
> {{}}}
>  
> it's just basically iterating over terms and don't do any sorting at all. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13156) Limiting field facet with certain terms via {!terms} not taking into account sorting

2019-01-24 Thread Mikhail Khludnev (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751050#comment-16751050
 ] 

Mikhail Khludnev commented on SOLR-13156:
-

mincount is a subject for further work, if there's no shortcut to handle it via 
common routine.
Also, enumerating terms out of order might have performance penalty, but again 
it's a separate topic.  

> Limiting field facet with certain terms via {!terms} not taking into account 
> sorting
> 
>
> Key: SOLR-13156
> URL: https://issues.apache.org/jira/browse/SOLR-13156
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Konstantin Perikov
>Priority: Major
>
> When I'm doing limiting facet keys with \{!terms} it doesn't take into 
> account sorting.
> First query not limiting the facet keys:
> {{facet.field=title=count=on=*:*}}
> Response as expected:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "book2",3, "book1",2, "book3",1]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
>  
> When doing it with limiting:
> {{facet.field=\{!terms=Book3,Book2,Book1}title=count=on=*:*}}
> I'm getting the exact order of how I list terms:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "Book3",1, "Book2",3, "Book1",2]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
> I've looked at the code, and it's clearly an issue there:
>  
> org.apache.solr.request.SimpleFacets#getListedTermCounts
>  
> {{for (String term : terms) {}}
> {{    int count = searcher.numDocs(ft.getFieldQuery(null, sf, term), 
> parsed.docs);}}
> {{    res.add(term, count);}}
> {{}}}
>  
> it's just basically iterating over terms and don't do any sorting at all. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-12999) Index replication could delete segments first

2019-01-24 Thread Noble Paul (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-12999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-12999:
-

  Assignee: Noble Paul
Attachment: SOLR-12999.patch

> Index replication could delete segments first
> -
>
> Key: SOLR-12999
> URL: https://issues.apache.org/jira/browse/SOLR-12999
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: replication (java)
>Reporter: David Smiley
>Assignee: Noble Paul
>Priority: Major
> Attachments: SOLR-12999.patch
>
>
> Index replication could optionally delete files that it knows will not be 
> needed _first_.  This would reduce disk capacity requirements of Solr, and it 
> would reduce some disk fragmentation when space get tight.
> Solr (IndexFetcher) already grabs the remote file list, and it could see 
> which files it has locally, then delete the others.  Today it asks Lucene to 
> {{deleteUnusedFiles}} at the end.  This new mode would probably only be 
> useful if there is no SolrIndexSearcher open, since it would prevent the 
> removal of files.
> The motivating scenario is a SolrCloud replica that is going into full 
> recovery.  It ought to not be fielding searches.  The code changes would not 
> depend on SolrCloud though.
> This option would have some danger the user should be aware of.  If the 
> replication fails, leaving the local files incomplete/corrupt, the only 
> recourse is to try full replication again.  You can't just give up and field 
> queries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13029) Allow HDFS backup/restore buffer size to be configured

2019-01-24 Thread Mikhail Khludnev (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751032#comment-16751032
 ] 

Mikhail Khludnev commented on SOLR-13029:
-

I like the patch, going to commit soon. Failed {{TestSimExtremeIndexing}} 
doesn't seem relevant. 

> Allow HDFS backup/restore buffer size to be configured
> --
>
> Key: SOLR-13029
> URL: https://issues.apache.org/jira/browse/SOLR-13029
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, hdfs
>Affects Versions: 7.5, 8.0
>Reporter: Tim Owen
>Priority: Major
> Attachments: SOLR-13029.patch, SOLR-13029.patch, SOLR-13029.patch
>
>
> There's a default hardcoded buffer size setting of 4096 in the HDFS code 
> which means in particular that restoring a backup from HDFS takes a long 
> time. Copying multi-GB files from HDFS using a buffer as small as 4096 bytes 
> is very inefficient. We changed this in our local build used in production to 
> 256kB and saw a 10x speed improvement when restoring a backup. Attached patch 
> simply makes this size configurable using a command line setting, much like 
> several other buffer size values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12121) JWT Authentication plugin

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751033#comment-16751033
 ] 

ASF subversion and git services commented on SOLR-12121:


Commit d09c724a0eaca3666dcb3fbc931bb64eb5f5b72f in lucene-solr's branch 
refs/heads/master from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d09c724 ]

SOLR-12121: Fix test fails in JWTAuthPluginIntegrationTest.testMetrics


> JWT Authentication plugin
> -
>
> Key: SOLR-12121
> URL: https://issues.apache.org/jira/browse/SOLR-12121
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Authentication
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: image-2018-08-27-13-04-04-183.png
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> A new Authentication plugin that will accept a [Json Web 
> Token|https://en.wikipedia.org/wiki/JSON_Web_Token] (JWT) in the 
> Authorization header and validate it by checking the cryptographic signature. 
> The plugin will not perform the authentication itself but assert that the 
> user was authenticated by the service that issued the JWT token.
> JWT defined a number of standard claims, and user principal can be fetched 
> from the {{sub}} (subject) claim and passed on to Solr. The plugin will 
> always check the {{exp}} (expiry) claim and optionally enforce checks on the 
> {{iss}} (issuer) and {{aud}} (audience) claims.
> The first version of the plugin will only support RSA signing keys and will 
> support fetching the public key of the issuer through a [Json Web 
> Key|https://tools.ietf.org/html/rfc7517] (JWK) file, either from a https URL 
> or from local file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13156) Limiting field facet with certain terms via {!terms} not taking into account sorting

2019-01-24 Thread Konstantin Perikov (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751027#comment-16751027
 ] 

Konstantin Perikov commented on SOLR-13156:
---

[~mkhludnev] [~ysee...@gmail.com]

I just figured out, that others features aren't working here as well (e.g. 
facet.mincount) Any ideas on this one?

 

> Limiting field facet with certain terms via {!terms} not taking into account 
> sorting
> 
>
> Key: SOLR-13156
> URL: https://issues.apache.org/jira/browse/SOLR-13156
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Konstantin Perikov
>Priority: Major
>
> When I'm doing limiting facet keys with \{!terms} it doesn't take into 
> account sorting.
> First query not limiting the facet keys:
> {{facet.field=title=count=on=*:*}}
> Response as expected:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "book2",3, "book1",2, "book3",1]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
>  
> When doing it with limiting:
> {{facet.field=\{!terms=Book3,Book2,Book1}title=count=on=*:*}}
> I'm getting the exact order of how I list terms:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "Book3",1, "Book2",3, "Book1",2]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
> I've looked at the code, and it's clearly an issue there:
>  
> org.apache.solr.request.SimpleFacets#getListedTermCounts
>  
> {{for (String term : terms) {}}
> {{    int count = searcher.numDocs(ft.getFieldQuery(null, sf, term), 
> parsed.docs);}}
> {{    res.add(term, count);}}
> {{}}}
>  
> it's just basically iterating over terms and don't do any sorting at all. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-13125) Optimize Queries when sorting by router.field

2019-01-24 Thread mosh (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mosh updated SOLR-13125:

Attachment: (was: SOLR-13125.patch)

> Optimize Queries when sorting by router.field
> -
>
> Key: SOLR-13125
> URL: https://issues.apache.org/jira/browse/SOLR-13125
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Minor
> Attachments: SOLR-13125-no-commit.patch, SOLR-13125.patch, 
> SOLR-13125.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We are currently testing TRA using Solr 7.7, having >300 shards in the alias, 
> with much growth in the coming months.
> The "hot" data(in our case, more recent) will be stored on stronger 
> nodes(SSD, more RAM, etc).
> A proposal of optimizing queries sorted by router.field(the field which TRA 
> uses to route the data to the correct collection) has emerged.
> Perhaps, in queries which are sorted by router.field, Solr could be smart 
> enough to wait for the more recent collections, and in case the limit was 
> reached cancel other queries(or just not block and wait for the results)?
> For example:
> When querying a TRA which with a filter on a different field than 
> router.field, but sorting by router.field desc, limit=100.
> Since this is a TRA, solr will issue queries for all the collections in the 
> alias.
> But to optimize this particular type of query, Solr could wait for the most 
> recent collection in the TRA, see whether the result set matches or exceeds 
> the limit. If so, the query could be returned to the user without waiting for 
> the rest of the shards. If not, the issuing node will block until the second 
> query returns, and so forth, until the limit of the request is reached.
> This might also be useful for deep paging, querying each collection and only 
> skipping to the next once there are no more results in the specified 
> collection.
> Thoughts or inputs are always welcome.
> This is just my two cents, and I'm always happy to brainstorm.
> Thanks in advance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-13125) Optimize Queries when sorting by router.field

2019-01-24 Thread mosh (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mosh updated SOLR-13125:

Attachment: SOLR-13125.patch

> Optimize Queries when sorting by router.field
> -
>
> Key: SOLR-13125
> URL: https://issues.apache.org/jira/browse/SOLR-13125
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Minor
> Attachments: SOLR-13125-no-commit.patch, SOLR-13125.patch, 
> SOLR-13125.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We are currently testing TRA using Solr 7.7, having >300 shards in the alias, 
> with much growth in the coming months.
> The "hot" data(in our case, more recent) will be stored on stronger 
> nodes(SSD, more RAM, etc).
> A proposal of optimizing queries sorted by router.field(the field which TRA 
> uses to route the data to the correct collection) has emerged.
> Perhaps, in queries which are sorted by router.field, Solr could be smart 
> enough to wait for the more recent collections, and in case the limit was 
> reached cancel other queries(or just not block and wait for the results)?
> For example:
> When querying a TRA which with a filter on a different field than 
> router.field, but sorting by router.field desc, limit=100.
> Since this is a TRA, solr will issue queries for all the collections in the 
> alias.
> But to optimize this particular type of query, Solr could wait for the most 
> recent collection in the TRA, see whether the result set matches or exceeds 
> the limit. If so, the query could be returned to the user without waiting for 
> the rest of the shards. If not, the issuing node will block until the second 
> query returns, and so forth, until the limit of the request is reached.
> This might also be useful for deep paging, querying each collection and only 
> skipping to the next once there are no more results in the specified 
> collection.
> Thoughts or inputs are always welcome.
> This is just my two cents, and I'm always happy to brainstorm.
> Thanks in advance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_172) - Build # 23567 - Failure!

2019-01-24 Thread Policeman Jenkins Server

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23567/
Java: 32bit/jdk1.8.0_172 -server -XX:+UseG1GC

All tests passed

Build Log:
[...truncated 59693 lines...]
-ecj-javadoc-lint-tests:
[mkdir] Created dir: /tmp/ecj816617752
 [ecj-lint] Compiling 942 source files to /tmp/ecj816617752
 [ecj-lint] invalid Class-Path header in manifest of jar file: 
/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
 [ecj-lint] invalid Class-Path header in manifest of jar file: 
/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
 [ecj-lint] --
 [ecj-lint] 1. WARNING in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test/org/apache/solr/analysis/TokenizerChainTest.java
 (at line 37)
 [ecj-lint] TokenizerChain tokenizerChain = new TokenizerChain(
 [ecj-lint]^^
 [ecj-lint] Resource leak: 'tokenizerChain' is never closed
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 2. WARNING in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test/org/apache/solr/cloud/ZkNodePropsTest.java
 (at line 48)
 [ecj-lint] new JavaBinCodec().marshal(zkProps.getProperties(), baos);
 [ecj-lint] ^^
 [ecj-lint] Resource leak: '' is never closed
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 3. ERROR in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test/org/apache/solr/cloud/autoscaling/NodeMarkersRegistrationTest.java
 (at line 31)
 [ecj-lint] import org.apache.lucene.util.LuceneTestCase.AwaitsFix;
 [ecj-lint]^^^
 [ecj-lint] The import org.apache.lucene.util.LuceneTestCase.AwaitsFix is never 
used
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 4. WARNING in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java
 (at line 716)
 [ecj-lint] throw new UnsupportedOperationException("must add at least 1 
node first");
 [ecj-lint] 
^^
 [ecj-lint] Resource leak: 'queryRequest' is not closed at this location
 [ecj-lint] --
 [ecj-lint] 5. WARNING in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java
 (at line 722)
 [ecj-lint] throw new UnsupportedOperationException("must add at least 1 
node first");
 [ecj-lint] 
^^
 [ecj-lint] Resource leak: 'queryRequest' is not closed at this location
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 6. WARNING in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java
 (at line 75)
 [ecj-lint] new JavaBinCodec(new BinaryResponseWriter.Resolver(null, 
null)).marshal(nl, baos);
 [ecj-lint] ^^^
 [ecj-lint] Resource leak: '' is never closed
 [ecj-lint] --
 [ecj-lint] 7. WARNING in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java
 (at line 77)
 [ecj-lint] nl = (NamedList) new JavaBinCodec().unmarshal(new 
ByteArrayInputStream(byteBuffer.array(), 0, byteBuffer.limit()));
 [ecj-lint]  ^^
 [ecj-lint] Resource leak: '' is never closed
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 8. WARNING in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test/org/apache/solr/update/processor/DistributedUpdateProcessorTest.java
 (at line 39)
 [ecj-lint] DistributedUpdateProcessor processor = new 
DistributedUpdateProcessor(
 [ecj-lint]^
 [ecj-lint] Resource leak: 'processor' is never closed
 [ecj-lint] --
 [ecj-lint] 8 problems (1 error, 7 warnings)

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:633: The following 
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:101: The following 
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build.xml:680: The 
following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:2099: 
The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:2132: 
Compile failed; see the compiler error output for details.

Total time: 85 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting 
ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting

[jira] [Commented] (LUCENE-8646) Add multi-term intervals

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-8646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751002#comment-16751002
 ] 

ASF subversion and git services commented on LUCENE-8646:
-

Commit 7d7ab14776b7257e09679d840182a4286928e452 in lucene-solr's branch 
refs/heads/master-deprecations from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7d7ab14 ]

LUCENE-8646: Multi-term intervals


> Add multi-term intervals
> 
>
> Key: LUCENE-8646
> URL: https://issues.apache.org/jira/browse/LUCENE-8646
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments: LUCENE-8646.patch
>
>
> We currently have no support for wildcard-type intervals.  I'd like to 
> explore adding some very basic support for prefix and wildcard interval 
> sources, but we need to ensure that we don't end up with the same performance 
> issues that dog SpanMultiTermQueryWrapper.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11998) RebalanceLeaders API broken response format with wt=JSON

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750998#comment-16750998
 ] 

ASF subversion and git services commented on SOLR-11998:


Commit 60aef389cfbe708701c50fc3a68a4082b69ec4d0 in lucene-solr's branch 
refs/heads/master-deprecations from Erick Erickson
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=60aef38 ]

SOLR-11998:RebalanceLeaders API broken response format with wt=JSON


> RebalanceLeaders API broken response format with wt=JSON
> 
>
> Key: SOLR-11998
> URL: https://issues.apache.org/jira/browse/SOLR-11998
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Erick Erickson
>Priority: Major
> Fix For: 8.0, 7.7, master (9.0)
>
> Attachments: SOLR-11998.patch, SOLR-11998.patch
>
>
> RebalanceLeaders has a weird looking JSON output because it uses NamedList 
> instead of SimpleOrderedMap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8638) Remove deprecated code in master

2019-01-24 Thread Alan Woodward (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742958#comment-16742958
 ] 

Alan Woodward edited comment on LUCENE-8638 at 1/24/19 10:41 AM:
-

The following files contain deprecated methods or classes. We can strike them 
off as they are fixed and pushed to the `master-deprecations` branch.

lucene/analysis/common/src/java/org/apache/lucene/analysis/synonym/SynonymFilter.java:110:@Deprecated
 
lucene/analysis/common/src/java/org/apache/lucene/analysis/synonym/SynonymFilterFactory.java:82:@Deprecated
-lucene/analysis/common/src/java/org/apache/lucene/analysis/util/FilesystemResourceLoader.java:55:
 @Deprecated-
-lucene/analysis/common/src/java/org/apache/lucene/analysis/util/ClasspathResourceLoader.java:44:
 @Deprecated-
 
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/FixBrokenOffsetsFilter.java:31:@Deprecated
 
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/WordDelimiterFilter.java:95:@Deprecated
 
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/WordDelimiterFilterFactory.java:58:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyBinaryDocValues.java:28:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacySortedSetDocValues.java:36:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyNumericDocValuesWrapper.java:30:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacySortedNumericDocValuesWrapper.java:29:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:47:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:76:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:105:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:152:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:207:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:271:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:319:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:383:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:438:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyDocValuesIterables.java:489:
 @Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacySortedSetDocValuesWrapper.java:30:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacySortedNumericDocValues.java:27:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacySortedDocValues.java:34:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyBinaryDocValuesWrapper.java:31:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacySortedDocValuesWrapper.java:30:@Deprecated
 
lucene/codecs/src/java/org/apache/lucene/codecs/memory/LegacyNumericDocValues.java:26:@Deprecated
 -lucene/core/src/test/org/apache/lucene/analysis/TestToken.java:31:@Deprecated-
 - undeprecated and moved to test-framework tests

-lucene/core/src/java/org/apache/lucene/util/SloppyMath.java:75: @Deprecated-
 -lucene/core/src/java/org/apache/lucene/util/IOUtils.java:426: @Deprecated-
 -lucene/core/src/java/org/apache/lucene/util/IOUtils.java:438: @Deprecated-
 lucene/core/src/java/org/apache/lucene/util/CharsRef.java:145: @Deprecated
 lucene/core/src/java/org/apache/lucene/util/CharsRef.java:149: @Deprecated
 lucene/core/src/java/org/apache/lucene/util/CharsRef.java:155: @Deprecated
 -lucene/core/src/java/org/apache/lucene/util/Version.java:39: @Deprecated-
 -lucene/core/src/java/org/apache/lucene/util/Version.java:75: @Deprecated-
 -lucene/core/src/java/org/apache/lucene/util/Constants.java:96: @Deprecated-
 -lucene/core/src/java/org/apache/lucene/util/Constants.java:103: @Deprecated-
 lucene/core/src/java/org/apache/lucene/search/FuzzyQuery.java:217: @Deprecated
 lucene/core/src/java/org/apache/lucene/search/FuzzyQuery.java:229: @Deprecated
 
lucene/core/src/java/org/apache/lucene/store/RAMOutputStream.java:37:@Deprecated
 lucene/core/src/java/org/apache/lucene/store/RAMInputStream.java:33:@Deprecated
 lucene/core/src/java/org/apache/lucene/store/RAMDirectory.java:57:@Deprecated
 lucene/core/src/java/org/apache/lucene/store/RAMFile.java:33:@Deprecated
 
-lucene/expressions/src/java/org/apache/lucene/expressions/js/JavascriptParser.java:43:
 @Deprecated-
 
-lucene/expressions/src/java/org/apache/lucene/expressions/js/JavascriptParser.java:60:
 @Deprecated-

[jira] [Commented] (SOLR-13156) Limiting field facet with certain terms via {!terms} not taking into account sorting

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751000#comment-16751000
 ] 

ASF subversion and git services commented on SOLR-13156:


Commit e68697a6de0b380665e9d2d787953035102c318c in lucene-solr's branch 
refs/heads/master-deprecations from Mikhail Khludnev
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e68697a ]

SOLR-13156: documenting functionality gap.


> Limiting field facet with certain terms via {!terms} not taking into account 
> sorting
> 
>
> Key: SOLR-13156
> URL: https://issues.apache.org/jira/browse/SOLR-13156
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Konstantin Perikov
>Priority: Major
>
> When I'm doing limiting facet keys with \{!terms} it doesn't take into 
> account sorting.
> First query not limiting the facet keys:
> {{facet.field=title=count=on=*:*}}
> Response as expected:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "book2",3, "book1",2, "book3",1]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
>  
> When doing it with limiting:
> {{facet.field=\{!terms=Book3,Book2,Book1}title=count=on=*:*}}
> I'm getting the exact order of how I list terms:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "Book3",1, "Book2",3, "Book1",2]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
> I've looked at the code, and it's clearly an issue there:
>  
> org.apache.solr.request.SimpleFacets#getListedTermCounts
>  
> {{for (String term : terms) {}}
> {{    int count = searcher.numDocs(ft.getFieldQuery(null, sf, term), 
> parsed.docs);}}
> {{    res.add(term, count);}}
> {{}}}
>  
> it's just basically iterating over terms and don't do any sorting at all. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8645) Add fixed field intervals

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751001#comment-16751001
 ] 

ASF subversion and git services commented on LUCENE-8645:
-

Commit 87d68c8253fcb928be4eb2b2d908393252a50ec5 in lucene-solr's branch 
refs/heads/master-deprecations from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=87d68c8 ]

LUCENE-8645: Intervals.fixField()


> Add fixed field intervals
> -
>
> Key: LUCENE-8645
> URL: https://issues.apache.org/jira/browse/LUCENE-8645
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments: LUCENE-8645.patch
>
>
> It can be useful to report intervals from one fields as if they came from 
> another.  For example, fast prefix searches can be implemented by indexing 
> text into two fields, one with the full terms and one with edge-ngrams 
> enabled; to do proximity searches against terms and prefixes, you could wrap 
> a term query against the ngrammed field so that its intervals appear to come 
> from the normal field, and use it an an ordered or unordered interval.
> This is analogous to the FieldMaskingSpanQuery, but has the advantage that we 
> don't use term statistics for scoring interval queries, so there is no issue 
> with mixing up field weights from different fields.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8649) LatLonShape: Within and disjoint queries don’t work with indexed multishapes

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750999#comment-16750999
 ] 

ASF subversion and git services commented on LUCENE-8649:
-

Commit 01dfe7bf4b2bd05326c66cc6297300f3dd321547 in lucene-solr's branch 
refs/heads/master-deprecations from Ignacio Vera
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=01dfe7b ]

LUCENE-8649: LatLonShape's within and disjoint queries can return false 
positives with indexed multi-shapes


> LatLonShape: Within and disjoint queries don’t work with indexed multishapes
> 
>
> Key: LUCENE-8649
> URL: https://issues.apache.org/jira/browse/LUCENE-8649
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/sandbox
>Reporter: Ignacio Vera
>Assignee: Ignacio Vera
>Priority: Major
> Fix For: 8.0, 7.7, master (9.0)
>
> Attachments: LUCENE-8649.patch, LUCENE-8649.patch, LUCENE-8649.patch, 
> LUCENE-8649.patch
>
>
> Within and disjoint queries return wrong results (false positives) when 
> querying for fields containing more than one shape. For example, a 
> multipolygon will return true for a within query if some of the polygons are 
> within and the other are disjoint. The same query will return true for 
> disjoint.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8657) CharsRef.compareTo() should always be in UTF-8 order

2019-01-24 Thread Alan Woodward (JIRA)

Alan Woodward created LUCENE-8657:
-

 Summary: CharsRef.compareTo() should always be in UTF-8 order
 Key: LUCENE-8657
 URL: https://issues.apache.org/jira/browse/LUCENE-8657
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Alan Woodward
Assignee: Alan Woodward
 Attachments: LUCENE-8657.patch

CharsRef.compareTo() currently directly compares byte values.  However, 
everywhere that CharsRef objects are compared in the codebase instead uses the 
deprecated UTF16SortedAsUTF8Comparator static comparator.  We should just 
reimplement compareTo() to use UTF-8 comparisons instead, and remove the 
deprecated methods.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8638) Remove deprecated code in master

2019-01-24 Thread Alan Woodward (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750983#comment-16750983
 ] 

Alan Woodward commented on LUCENE-8638:
---

I opened LUCENE-8657 for deprecations in CharsRef

> Remove deprecated code in master
> 
>
> Key: LUCENE-8638
> URL: https://issues.apache.org/jira/browse/LUCENE-8638
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: master (9.0)
>
>
> There are a number of deprecations in master that should be removed. This 
> issue is to keep track of deprecations as a whole, some individual 
> deprecations may require their own issues.
>  
> Work on this issue should be pushed to the `master-deprecations` branch on 
> gitbox



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8657) CharsRef.compareTo() should always be in UTF-8 order

2019-01-24 Thread Alan Woodward (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750978#comment-16750978
 ] 

Alan Woodward commented on LUCENE-8657:
---

Here is a patch.  The only minor difference I can see this making is that there 
is a possibility that the Hunspell token filter might emit stacked tokens in a 
slightly different order, but I don't think that will be an issue anywhere.

> CharsRef.compareTo() should always be in UTF-8 order
> 
>
> Key: LUCENE-8657
> URL: https://issues.apache.org/jira/browse/LUCENE-8657
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8657.patch
>
>
> CharsRef.compareTo() currently directly compares byte values.  However, 
> everywhere that CharsRef objects are compared in the codebase instead uses 
> the deprecated UTF16SortedAsUTF8Comparator static comparator.  We should just 
> reimplement compareTo() to use UTF-8 comparisons instead, and remove the 
> deprecated methods.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8657) CharsRef.compareTo() should always be in UTF-8 order

2019-01-24 Thread Alan Woodward (JIRA)



 [ 
https://issues.apache.org/jira/browse/LUCENE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8657:
--
Fix Version/s: 8.0

> CharsRef.compareTo() should always be in UTF-8 order
> 
>
> Key: LUCENE-8657
> URL: https://issues.apache.org/jira/browse/LUCENE-8657
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments: LUCENE-8657.patch
>
>
> CharsRef.compareTo() currently directly compares byte values.  However, 
> everywhere that CharsRef objects are compared in the codebase instead uses 
> the deprecated UTF16SortedAsUTF8Comparator static comparator.  We should just 
> reimplement compareTo() to use UTF-8 comparisons instead, and remove the 
> deprecated methods.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13156) Limiting field facet with certain terms via {!terms} not taking into account sorting

2019-01-24 Thread Konstantin Perikov (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750975#comment-16750975
 ] 

Konstantin Perikov commented on SOLR-13156:
---

[~superman369] I hope to provide patch relatively quick, however I have no idea 
how long it would take to go through the review/release process, etc.

> Limiting field facet with certain terms via {!terms} not taking into account 
> sorting
> 
>
> Key: SOLR-13156
> URL: https://issues.apache.org/jira/browse/SOLR-13156
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Konstantin Perikov
>Priority: Major
>
> When I'm doing limiting facet keys with \{!terms} it doesn't take into 
> account sorting.
> First query not limiting the facet keys:
> {{facet.field=title=count=on=*:*}}
> Response as expected:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "book2",3, "book1",2, "book3",1]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
>  
> When doing it with limiting:
> {{facet.field=\{!terms=Book3,Book2,Book1}title=count=on=*:*}}
> I'm getting the exact order of how I list terms:
> {{"facet_counts":\{ "facet_queries":{}, "facet_fields":\{ "title":[ 
> "Book3",1, "Book2",3, "Book1",2]}, "facet_ranges":{}, "facet_intervals":{}, 
> "facet_heatmaps":{}
> I've looked at the code, and it's clearly an issue there:
>  
> org.apache.solr.request.SimpleFacets#getListedTermCounts
>  
> {{for (String term : terms) {}}
> {{    int count = searcher.numDocs(ft.getFieldQuery(null, sf, term), 
> parsed.docs);}}
> {{    res.add(term, count);}}
> {{}}}
>  
> it's just basically iterating over terms and don't do any sorting at all. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8657) CharsRef.compareTo() should always be in UTF-8 order

2019-01-24 Thread Alan Woodward (JIRA)



 [ 
https://issues.apache.org/jira/browse/LUCENE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8657:
--
Attachment: LUCENE-8657.patch

> CharsRef.compareTo() should always be in UTF-8 order
> 
>
> Key: LUCENE-8657
> URL: https://issues.apache.org/jira/browse/LUCENE-8657
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8657.patch
>
>
> CharsRef.compareTo() currently directly compares byte values.  However, 
> everywhere that CharsRef objects are compared in the codebase instead uses 
> the deprecated UTF16SortedAsUTF8Comparator static comparator.  We should just 
> reimplement compareTo() to use UTF-8 comparisons instead, and remove the 
> deprecated methods.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13072) Management of markers for nodeLost / nodeAdded events is broken

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750951#comment-16750951
 ] 

ASF subversion and git services commented on SOLR-13072:


Commit 84819c837955c163f5a1a6203ed8005f22053fa8 in lucene-solr's branch 
refs/heads/master from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=84819c8 ]

SOLR-13072: Fix precommit.


> Management of markers for nodeLost / nodeAdded events is broken
> ---
>
> Key: SOLR-13072
> URL: https://issues.apache.org/jira/browse/SOLR-13072
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 7.5, 7.6, 8.0
>Reporter: Andrzej Bialecki 
>Assignee: Andrzej Bialecki 
>Priority: Major
> Fix For: 8.0, 7.7, master (9.0)
>
>
> In order to prevent {{nodeLost}} events from being lost when it's the 
> Overseer leader that is the node that was lost a mechanism was added to 
> record markers for these events by any other live node, in 
> {{ZkController.registerLiveNodesListener()}}. As similar mechanism also 
> exists for {{nodeAdded}} events.
> On Overseer leader restart if the autoscaling configuration didn't contain 
> any triggers that consume {{nodeLost}} events then these markers are removed. 
> If there are 1 or more trigger configs that consume {{nodeLost}} events then 
> these triggers would read the markers, remove them and generate appropriate 
> events.
> However, as the {{NodeMarkersRegistrationTest}} shows this mechanism is 
> broken and susceptible to race conditions.
> It's not unusual to have more than 1 {{nodeLost}} trigger because in addition 
> to any user-defined triggers there's always one that is automatically defined 
> if missing: {{.auto_add_replicas}}. However, if there's more than 1 
> {{nodeLost}} trigger then the process of consuming and removing the markers 
> becomes non-deterministic - each trigger may pick up (and delete) all, none, 
> or some of the markers.
> So as it is now this mechanism is broken if more than 1 {{nodeLost}} or more 
> than 1 {{nodeAdded}} trigger is defined.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13029) Allow HDFS backup/restore buffer size to be configured

2019-01-24 Thread Lucene/Solr QA (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750953#comment-16750953
 ] 

Lucene/Solr QA commented on SOLR-13029:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  4m 
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  2m  5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  2m  5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  2m  5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 54s{color} 
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m  9s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.autoscaling.sim.TestSimExtremeIndexing |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13029 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12955939/SOLR-13029.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP 
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / 72a99e9 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | 1.8.0_191 |
| unit | 
https://builds.apache.org/job/PreCommit-SOLR-Build/266/artifact/out/patch-unit-solr_core.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/266/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/266/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Allow HDFS backup/restore buffer size to be configured
> --
>
> Key: SOLR-13029
> URL: https://issues.apache.org/jira/browse/SOLR-13029
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, hdfs
>Affects Versions: 7.5, 8.0
>Reporter: Tim Owen
>Priority: Major
> Attachments: SOLR-13029.patch, SOLR-13029.patch, SOLR-13029.patch
>
>
> There's a default hardcoded buffer size setting of 4096 in the HDFS code 
> which means in particular that restoring a backup from HDFS takes a long 
> time. Copying multi-GB files from HDFS using a buffer as small as 4096 bytes 
> is very inefficient. We changed this in our local build used in production to 
> 256kB and saw a 10x speed improvement when restoring a backup. Attached patch 
> simply makes this size configurable using a command line setting, much like 
> several other buffer size values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13072) Management of markers for nodeLost / nodeAdded events is broken

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750948#comment-16750948
 ] 

ASF subversion and git services commented on SOLR-13072:


Commit 6f3d8a97706e1d5dca6b32de6e03953482315cdc in lucene-solr's branch 
refs/heads/branch_8x from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6f3d8a9 ]

SOLR-13072: Fix precommit.


> Management of markers for nodeLost / nodeAdded events is broken
> ---
>
> Key: SOLR-13072
> URL: https://issues.apache.org/jira/browse/SOLR-13072
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 7.5, 7.6, 8.0
>Reporter: Andrzej Bialecki 
>Assignee: Andrzej Bialecki 
>Priority: Major
> Fix For: 8.0, 7.7, master (9.0)
>
>
> In order to prevent {{nodeLost}} events from being lost when it's the 
> Overseer leader that is the node that was lost a mechanism was added to 
> record markers for these events by any other live node, in 
> {{ZkController.registerLiveNodesListener()}}. As similar mechanism also 
> exists for {{nodeAdded}} events.
> On Overseer leader restart if the autoscaling configuration didn't contain 
> any triggers that consume {{nodeLost}} events then these markers are removed. 
> If there are 1 or more trigger configs that consume {{nodeLost}} events then 
> these triggers would read the markers, remove them and generate appropriate 
> events.
> However, as the {{NodeMarkersRegistrationTest}} shows this mechanism is 
> broken and susceptible to race conditions.
> It's not unusual to have more than 1 {{nodeLost}} trigger because in addition 
> to any user-defined triggers there's always one that is automatically defined 
> if missing: {{.auto_add_replicas}}. However, if there's more than 1 
> {{nodeLost}} trigger then the process of consuming and removing the markers 
> becomes non-deterministic - each trigger may pick up (and delete) all, none, 
> or some of the markers.
> So as it is now this mechanism is broken if more than 1 {{nodeLost}} or more 
> than 1 {{nodeAdded}} trigger is defined.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13072) Management of markers for nodeLost / nodeAdded events is broken

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750943#comment-16750943
 ] 

ASF subversion and git services commented on SOLR-13072:


Commit aadce4f409f9b5105510a9e82c80a7e55bae46e2 in lucene-solr's branch 
refs/heads/branch_7x from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=aadce4f ]

SOLR-13072: Fix precommit.


> Management of markers for nodeLost / nodeAdded events is broken
> ---
>
> Key: SOLR-13072
> URL: https://issues.apache.org/jira/browse/SOLR-13072
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 7.5, 7.6, 8.0
>Reporter: Andrzej Bialecki 
>Assignee: Andrzej Bialecki 
>Priority: Major
> Fix For: 8.0, 7.7, master (9.0)
>
>
> In order to prevent {{nodeLost}} events from being lost when it's the 
> Overseer leader that is the node that was lost a mechanism was added to 
> record markers for these events by any other live node, in 
> {{ZkController.registerLiveNodesListener()}}. As similar mechanism also 
> exists for {{nodeAdded}} events.
> On Overseer leader restart if the autoscaling configuration didn't contain 
> any triggers that consume {{nodeLost}} events then these markers are removed. 
> If there are 1 or more trigger configs that consume {{nodeLost}} events then 
> these triggers would read the markers, remove them and generate appropriate 
> events.
> However, as the {{NodeMarkersRegistrationTest}} shows this mechanism is 
> broken and susceptible to race conditions.
> It's not unusual to have more than 1 {{nodeLost}} trigger because in addition 
> to any user-defined triggers there's always one that is automatically defined 
> if missing: {{.auto_add_replicas}}. However, if there's more than 1 
> {{nodeLost}} trigger then the process of consuming and removing the markers 
> becomes non-deterministic - each trigger may pick up (and delete) all, none, 
> or some of the markers.
> So as it is now this mechanism is broken if more than 1 {{nodeLost}} or more 
> than 1 {{nodeAdded}} trigger is defined.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5698) Evaluate Lucene classification on publicly available datasets

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750916#comment-16750916
 ] 

ASF subversion and git services commented on LUCENE-5698:
-

Commit 6df32fbc35c43077a0275c67fa86139c36a6d110 in lucene-solr's branch 
refs/heads/master from Tommaso Teofili
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6df32fb ]

LUCENE-5698 - forbidden APIs related fixes


> Evaluate Lucene classification on publicly available datasets
> -
>
> Key: LUCENE-5698
> URL: https://issues.apache.org/jira/browse/LUCENE-5698
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: modules/classification
>Reporter: Gergő Törcsvári
>Assignee: Tommaso Teofili
>Priority: Major
> Attachments: 0803-test.patch, 0810-test.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The Lucene classification module need some publicly available dataset for 
> keep track on the development.
> Now it woud be nice to have some generated fast test-sets, and some bigger 
> real world dataset too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5698) Evaluate Lucene classification on publicly available datasets

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750914#comment-16750914
 ] 

ASF subversion and git services commented on LUCENE-5698:
-

Commit 552c367c5d2f060c758fb54ae97a0e8d4d8e60b7 in lucene-solr's branch 
refs/heads/master from Tommaso Teofili
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=552c367 ]

LUCENE-5698 - added test for 20n dataset, minor code adjustments


> Evaluate Lucene classification on publicly available datasets
> -
>
> Key: LUCENE-5698
> URL: https://issues.apache.org/jira/browse/LUCENE-5698
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: modules/classification
>Reporter: Gergő Törcsvári
>Assignee: Tommaso Teofili
>Priority: Major
> Attachments: 0803-test.patch, 0810-test.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The Lucene classification module need some publicly available dataset for 
> keep track on the development.
> Now it woud be nice to have some generated fast test-sets, and some bigger 
> real world dataset too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-8.x - Build # 15 - Failure

2019-01-24 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Tests-8.x/15/

All tests passed

Build Log:
[...truncated 61419 lines...]
-ecj-javadoc-lint-tests:
[mkdir] Created dir: /tmp/ecj1726967574
 [ecj-lint] Compiling 940 source files to /tmp/ecj1726967574
 [ecj-lint] invalid Class-Path header in manifest of jar file: 
/x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
 [ecj-lint] invalid Class-Path header in manifest of jar file: 
/x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
 [ecj-lint] --
 [ecj-lint] 1. WARNING in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/core/src/test/org/apache/solr/analysis/TokenizerChainTest.java
 (at line 37)
 [ecj-lint] TokenizerChain tokenizerChain = new TokenizerChain(
 [ecj-lint]^^
 [ecj-lint] Resource leak: 'tokenizerChain' is never closed
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 2. WARNING in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/core/src/test/org/apache/solr/cloud/ZkNodePropsTest.java
 (at line 48)
 [ecj-lint] new JavaBinCodec().marshal(zkProps.getProperties(), baos);
 [ecj-lint] ^^
 [ecj-lint] Resource leak: '' is never closed
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 3. ERROR in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/core/src/test/org/apache/solr/cloud/autoscaling/NodeMarkersRegistrationTest.java
 (at line 31)
 [ecj-lint] import org.apache.lucene.util.LuceneTestCase.AwaitsFix;
 [ecj-lint]^^^
 [ecj-lint] The import org.apache.lucene.util.LuceneTestCase.AwaitsFix is never 
used
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 4. WARNING in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/core/src/test/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java
 (at line 716)
 [ecj-lint] throw new UnsupportedOperationException("must add at least 1 
node first");
 [ecj-lint] 
^^
 [ecj-lint] Resource leak: 'queryRequest' is not closed at this location
 [ecj-lint] --
 [ecj-lint] 5. WARNING in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/core/src/test/org/apache/solr/cloud/autoscaling/sim/SimCloudManager.java
 (at line 722)
 [ecj-lint] throw new UnsupportedOperationException("must add at least 1 
node first");
 [ecj-lint] 
^^
 [ecj-lint] Resource leak: 'queryRequest' is not closed at this location
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 6. WARNING in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java
 (at line 75)
 [ecj-lint] new JavaBinCodec(new BinaryResponseWriter.Resolver(null, 
null)).marshal(nl, baos);
 [ecj-lint] ^^^
 [ecj-lint] Resource leak: '' is never closed
 [ecj-lint] --
 [ecj-lint] 7. WARNING in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java
 (at line 77)
 [ecj-lint] nl = (NamedList) new JavaBinCodec().unmarshal(new 
ByteArrayInputStream(byteBuffer.array(), 0, byteBuffer.limit()));
 [ecj-lint]  ^^
 [ecj-lint] Resource leak: '' is never closed
 [ecj-lint] --
 [ecj-lint] --
 [ecj-lint] 8. WARNING in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/core/src/test/org/apache/solr/update/processor/DistributedUpdateProcessorTest.java
 (at line 39)
 [ecj-lint] DistributedUpdateProcessor processor = new 
DistributedUpdateProcessor(
 [ecj-lint]^
 [ecj-lint] Resource leak: 'processor' is never closed
 [ecj-lint] --
 [ecj-lint] 8 problems (1 error, 7 warnings)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/build.xml:633: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/build.xml:101: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/solr/build.xml:680: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/lucene/common-build.xml:2099:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-8.x/lucene/common-build.xml:2132:
 Compile failed; see the compiler error output for details.

Total time: 99 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[jira] [Commented] (LUCENE-5698) Evaluate Lucene classification on publicly available datasets

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750920#comment-16750920
 ] 

ASF subversion and git services commented on LUCENE-5698:
-

Commit a557aa0aa3a807bb29dc01101bab4d54ec6fba4e in lucene-solr's branch 
refs/heads/master from Tommaso Teofili
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a557aa0 ]

Merge pull request #544 from tteofili/LUCENE-5698

LUCENE-5698 - added test to benchmark lucene classification against 20n dataset

> Evaluate Lucene classification on publicly available datasets
> -
>
> Key: LUCENE-5698
> URL: https://issues.apache.org/jira/browse/LUCENE-5698
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: modules/classification
>Reporter: Gergő Törcsvári
>Assignee: Tommaso Teofili
>Priority: Major
> Attachments: 0803-test.patch, 0810-test.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The Lucene classification module need some publicly available dataset for 
> keep track on the development.
> Now it woud be nice to have some generated fast test-sets, and some bigger 
> real world dataset too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5698) Evaluate Lucene classification on publicly available datasets

2019-01-24 Thread Tommaso Teofili (JIRA)



 [ 
https://issues.apache.org/jira/browse/LUCENE-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili resolved LUCENE-5698.
-
   Resolution: Fixed
Fix Version/s: trunk

> Evaluate Lucene classification on publicly available datasets
> -
>
> Key: LUCENE-5698
> URL: https://issues.apache.org/jira/browse/LUCENE-5698
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: modules/classification
>Reporter: Gergő Törcsvári
>Assignee: Tommaso Teofili
>Priority: Major
> Fix For: trunk
>
> Attachments: 0803-test.patch, 0810-test.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The Lucene classification module need some publicly available dataset for 
> keep track on the development.
> Now it woud be nice to have some generated fast test-sets, and some bigger 
> real world dataset too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5698) Evaluate Lucene classification on publicly available datasets

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750919#comment-16750919
 ] 

ASF subversion and git services commented on LUCENE-5698:
-

Commit a557aa0aa3a807bb29dc01101bab4d54ec6fba4e in lucene-solr's branch 
refs/heads/master from Tommaso Teofili
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a557aa0 ]

Merge pull request #544 from tteofili/LUCENE-5698

LUCENE-5698 - added test to benchmark lucene classification against 20n dataset

> Evaluate Lucene classification on publicly available datasets
> -
>
> Key: LUCENE-5698
> URL: https://issues.apache.org/jira/browse/LUCENE-5698
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: modules/classification
>Reporter: Gergő Törcsvári
>Assignee: Tommaso Teofili
>Priority: Major
> Attachments: 0803-test.patch, 0810-test.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The Lucene classification module need some publicly available dataset for 
> keep track on the development.
> Now it woud be nice to have some generated fast test-sets, and some bigger 
> real world dataset too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5698) Evaluate Lucene classification on publicly available datasets

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750917#comment-16750917
 ] 

ASF subversion and git services commented on LUCENE-5698:
-

Commit 33896dde5a6017d42ef45a115c130f3b29d5496f in lucene-solr's branch 
refs/heads/master from Tommaso Teofili
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=33896dd ]

LUCENE-5698 - minor adjustments


> Evaluate Lucene classification on publicly available datasets
> -
>
> Key: LUCENE-5698
> URL: https://issues.apache.org/jira/browse/LUCENE-5698
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: modules/classification
>Reporter: Gergő Törcsvári
>Assignee: Tommaso Teofili
>Priority: Major
> Attachments: 0803-test.patch, 0810-test.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The Lucene classification module need some publicly available dataset for 
> keep track on the development.
> Now it woud be nice to have some generated fast test-sets, and some bigger 
> real world dataset too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5698) Evaluate Lucene classification on publicly available datasets

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750918#comment-16750918
 ] 

ASF subversion and git services commented on LUCENE-5698:
-

Commit c32e9b6c0a564fffbe4c59e172a7b3dbe5d8f0df in lucene-solr's branch 
refs/heads/master from Tommaso Teofili
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c32e9b6 ]

Merge branch 'master' of github.com:apache/lucene-solr into LUCENE-5698


> Evaluate Lucene classification on publicly available datasets
> -
>
> Key: LUCENE-5698
> URL: https://issues.apache.org/jira/browse/LUCENE-5698
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: modules/classification
>Reporter: Gergő Törcsvári
>Assignee: Tommaso Teofili
>Priority: Major
> Attachments: 0803-test.patch, 0810-test.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The Lucene classification module need some publicly available dataset for 
> keep track on the development.
> Now it woud be nice to have some generated fast test-sets, and some bigger 
> real world dataset too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5698) Evaluate Lucene classification on publicly available datasets

2019-01-24 Thread ASF subversion and git services (JIRA)



[ 
https://issues.apache.org/jira/browse/LUCENE-5698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750915#comment-16750915
 ] 

ASF subversion and git services commented on LUCENE-5698:
-

Commit e32170db833faaf93436210a5192f44410d573cb in lucene-solr's branch 
refs/heads/master from Tommaso Teofili
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e32170d ]

Merge branch 'master' of github.com:apache/lucene-solr into LUCENE-5698


> Evaluate Lucene classification on publicly available datasets
> -
>
> Key: LUCENE-5698
> URL: https://issues.apache.org/jira/browse/LUCENE-5698
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: modules/classification
>Reporter: Gergő Törcsvári
>Assignee: Tommaso Teofili
>Priority: Major
> Attachments: 0803-test.patch, 0810-test.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The Lucene classification module need some publicly available dataset for 
> keep track on the development.
> Now it woud be nice to have some generated fast test-sets, and some bigger 
> real world dataset too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] tteofili merged pull request #544: LUCENE-5698 - added test for 20n dataset, minor code adjustments

2019-01-24 Thread GitBox

tteofili merged pull request #544: LUCENE-5698 - added test for 20n dataset, 
minor code adjustments
URL: https://github.com/apache/lucene-solr/pull/544
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13125) Optimize Queries when sorting by router.field

2019-01-24 Thread mosh (JIRA)



[ 
https://issues.apache.org/jira/browse/SOLR-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750904#comment-16750904
 ] 

mosh commented on SOLR-13125:
-

Uploaded a new patch which replaces the use of Collections#sort with 
ReverseListIterator, for better performance.

> Optimize Queries when sorting by router.field
> -
>
> Key: SOLR-13125
> URL: https://issues.apache.org/jira/browse/SOLR-13125
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Minor
> Attachments: SOLR-13125-no-commit.patch, SOLR-13125.patch, 
> SOLR-13125.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We are currently testing TRA using Solr 7.7, having >300 shards in the alias, 
> with much growth in the coming months.
> The "hot" data(in our case, more recent) will be stored on stronger 
> nodes(SSD, more RAM, etc).
> A proposal of optimizing queries sorted by router.field(the field which TRA 
> uses to route the data to the correct collection) has emerged.
> Perhaps, in queries which are sorted by router.field, Solr could be smart 
> enough to wait for the more recent collections, and in case the limit was 
> reached cancel other queries(or just not block and wait for the results)?
> For example:
> When querying a TRA which with a filter on a different field than 
> router.field, but sorting by router.field desc, limit=100.
> Since this is a TRA, solr will issue queries for all the collections in the 
> alias.
> But to optimize this particular type of query, Solr could wait for the most 
> recent collection in the TRA, see whether the result set matches or exceeds 
> the limit. If so, the query could be returned to the user without waiting for 
> the rest of the shards. If not, the issuing node will block until the second 
> query returns, and so forth, until the limit of the request is reached.
> This might also be useful for deep paging, querying each collection and only 
> skipping to the next once there are no more results in the specified 
> collection.
> Thoughts or inputs are always welcome.
> This is just my two cents, and I'm always happy to brainstorm.
> Thanks in advance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-13125) Optimize Queries when sorting by router.field

2019-01-24 Thread mosh (JIRA)



 [ 
https://issues.apache.org/jira/browse/SOLR-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mosh updated SOLR-13125:

Attachment: SOLR-13125.patch

> Optimize Queries when sorting by router.field
> -
>
> Key: SOLR-13125
> URL: https://issues.apache.org/jira/browse/SOLR-13125
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Minor
> Attachments: SOLR-13125-no-commit.patch, SOLR-13125.patch, 
> SOLR-13125.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We are currently testing TRA using Solr 7.7, having >300 shards in the alias, 
> with much growth in the coming months.
> The "hot" data(in our case, more recent) will be stored on stronger 
> nodes(SSD, more RAM, etc).
> A proposal of optimizing queries sorted by router.field(the field which TRA 
> uses to route the data to the correct collection) has emerged.
> Perhaps, in queries which are sorted by router.field, Solr could be smart 
> enough to wait for the more recent collections, and in case the limit was 
> reached cancel other queries(or just not block and wait for the results)?
> For example:
> When querying a TRA which with a filter on a different field than 
> router.field, but sorting by router.field desc, limit=100.
> Since this is a TRA, solr will issue queries for all the collections in the 
> alias.
> But to optimize this particular type of query, Solr could wait for the most 
> recent collection in the TRA, see whether the result set matches or exceeds 
> the limit. If so, the query could be returned to the user without waiting for 
> the rest of the shards. If not, the issuing node will block until the second 
> query returns, and so forth, until the limit of the request is reached.
> This might also be useful for deep paging, querying each collection and only 
> skipping to the next once there are no more results in the specified 
> collection.
> Thoughts or inputs are always welcome.
> This is just my two cents, and I'm always happy to brainstorm.
> Thanks in advance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Size Limitation upto 32 kb limitation

2019-01-24 Thread Jan Høydahl

I cannot see why you'd want a single term of 32kb in your index anyway. Can you 
give us examples of what these terms are and how you will search them?
What kind of files are you indexing, could it be like bad PDFs consisting of a 
bunch of binary garbage?
Try adding a lengthFilterFactory to your fieldType(s). See 
https://lucene.apache.org/solr/guide/7_6/filter-descriptions.html#length-filter 


--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 24. jan. 2019 kl. 06:51 skrev Kranthi Kumar K 
> :
> 
> Thank you Bernd Fehling for your suggested solution, I've tried the same by 
> changing the type and added multivalued to true in Schema.xml file i.e,
> change from: 
>  
> 
>  
> Changed to: 
>  
>  multiValued="true" />
>  
> After changing it also still we are unable to import the files size > 32 kb. 
> please find the solution suggested by Bernd in the below url:
>  
> http://lucene.472066.n3.nabble.com/Re-Solr-Size-Limitation-upto-32-kb-limitation-td4421569.html
>  
> 
>  
> Bernd Fehling, could you please suggest another alternative solution to 
> resolve our issue, which would help us alot?
>  
> Please let me know for any questions.
>  
> 
> 
> Thanks & Regards,
> Kranthi Kumar.K,
> Software Engineer,
> Ccube Fintech Global Services Pvt Ltd.,
> Email/Skype: kranthikuma...@ccubefintech.com 
> ,
> Mobile: +91-8978078449.
>  
>  
> From: Kranthi Kumar K 
> Sent: Friday, January 18, 2019 4:22 PM
> To: dev@lucene.apache.org ; 
> solr-u...@lucene.apache.org 
> Cc: Ananda Babu medida  >; Srinivasa Reddy Karri 
>  >; Michelle Ngo 
> mailto:michelle@ccube.com.au>>; Ravi Vangala 
> mailto:ravi.vang...@ccubefintech.com>>
> Subject: RE: Solr Size Limitation upto 32 kb limitation
>  
> Hi team,
>  
> Thank you Erick Erickson ,Bernd Fehling , Jan Hoydahl for your suggested 
> solutions. I’ve tried the suggested one’s and still we are unable to import 
> files havingsize  >32 kb, it is displaying same error.
>  
> Below link has the suggested solutions. Please have a look once.
>  
> http://lucene.472066.n3.nabble.com/Solr-Size-Limitation-upto-32-KB-files-td4419779.html
>  
> 
>  
> As per Erick Erickson, I’ve changed the string type to Text type based and 
> still the issue occurs .
> I’ve changed from :
>  
> 
>  
> Changed to:
>  
> 
>  
> If we do so, it is showing error in the log, please find the error in the 
> attachment.
>  
> If I change to:
>  
> 
>  
> It is not showing any error , but the issue still exists.
>  
> As per Jan Hoydahl, I have gone through the link that you have provided and 
> checked ‘requestParsers’ tag in solrconfig.xml,
>  
> RequestParsers tag in our application is as follows:
>  
> ‘ multipartUploadLimitInKB="2048000"
> formdataUploadLimitInKB="2048"
> addHttpRequestToContext="false"/>’
> Request parsers, which we are using and in the link you have provided are 
> similar. And still we are unable to import the files size >32 kb.
>  
> As per Bernd Fehling, we are using Solr 4.10.2. you have mentioned as,
> ‘If you are trying to add larger content then you have to "chop" that 
> by yourself and add it as multivalued. Can be done within a self written 
> loader. ’
>  
> I’m a newbie to Solr and I didn’t get what exactly ‘self written loader’ is?
>  
> Could you please provide us sample code, that helps us to go further?
>  
>  
> 
> 
> Thanks & Regards,
> Kranthi Kumar.K,
> Software Engineer,
> Ccube Fintech Global Services Pvt Ltd.,
> Email/Skype: kranthikuma...@ccubefintech.com 
> ,
> Mobile: +91-8978078449.
>  
>  
> From: Kranthi Kumar K  > 
> Sent: Thursday, January 17, 2019 12:43 PM
> To: dev@lucene.apache.org ; 
> solr-u...@lucene.apache.org 
> Cc: Ananda Babu medida  >; Srinivasa Reddy Karri 
>  >; Michelle Ngo 
> mailto:michelle@ccube.com.au>>
> Subject: Re: Solr Size Limitation upto 32 kb limitation
>  
> Hi Team,
> 
>  
> 
> Can we have any updates on the below issue? We are awaiting your reply.
> 
>  
> 
> Thanks,
> 
> Kranthi kumar.K
> 
> From: Kranthi Kumar K
> Sent: Friday, January 4, 2019 5:01:38 PM
> To: dev@lucene.apache.org 
> Cc: Ananda Babu medida; Srinivasa Reddy Karri
> Subject: Solr Size Limitation upto 32 kb limitation
>  
> Hi team,
> 
>

73 matches

Mail list logo